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Abstract of a Major Applied Research Project Presented to 
Nova University in Partial Fulfillment of the 
Requirements for the Degree of 
Doctor of Education 

RELATIONSHIPS OF STUDENT RATINGS OF COURSE, INSTRUCTOR, AND STUDENT 
ELEMENTS TO STUDENT OVERALL RATINGS OF INSTRUCTORS OF 
COURSES AT LAKESHORE TECHNICAL COLLEGE 

by 

Marvin A. Schrader 
July 1988 

Lakeshore Technical College has approximately one hundred 
full-time instructors. Instructional administrators are responsible 
for implementing the instructor growth program which had been loosely 
organized in previous years. During the past several years, a need 
became apparent for a more structured, yet flexible, . program to ensure 
a capable, enthusiastic, and technically competent instructional staff. 

Several instructional management staff members developed the 
Lifelong Investment For Excellence (LI^E) program to assist in indi- 
vidual instructor professional growth. Goals for professional growth 
are developed and ranked to a large extent on student assessment of 
instructor's performance. The major component in this assessment is 
the result of analv;is of student ratings for particular instructor 
evaluation elements. The analysis involves student ratings of a number 
of particular course elements (e.g., difficulty and type), instructor 
elements (e.g., enthusiasm and explanations), and student elements 
(e.g., age and grade expected). 
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The purpose of this study was to refine a part of the instruc- 
tional assessment procedure at LTC. This was accomplished by obtaining 
and analyzing data for course, instructor, and student elements to 
determine their relative relationships to student overall ratings of 
instruct! s. Equally important, this study also determined the biasing 
effects of particular course, instructor, and student elements; :tudent 
demographics; and college course and program classifications on student 
overall ratings of instructors. 

The research questions developed for this study were: 

1. Is there a relationship evident between student overall 
ratings of instructors and student ratings of the three major groupings 
of elements (course, instructor, and student) in the instructor assess- 
ment at LTC? If so, what is the nature of the relationship? 

2. Is there a relationship between student overall ratings of 
instructors and student ratings of particular items within each of the 
three major element groups, student demographic characteristics, or the 
place of the course in the college's academic program? 

The sample included 262 first-year Lakeshore Technical College 
students enrolled in occupational programs having at least one general 
education course and one occupational -specific course in the first year 
of the program offered during the 1986-87 fall semester. Students in 
twenty-four courses in eighteen programs were included in the study. 
The Student Assessment of Instruction instrument, developed at the 
college during 1984-86, was used to obtain student rating data. The 
student rating data were analyzed using the regression analysis to 
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obtain the answer to the first question and One-Way Analysis of the 
Variance (ANOVA) to obtain the answer to the second question. 

The answers provided the information necessary to provide a 
basis for adjusting the student ratings and for determining the rela- 
tionship of student ratings to particular elements with their overall 
ratings of instructors. Staff at the college will have a basis for 
adjusting the student ratings to ensure comparability regardless of the 
particular course and students involved. The adjusted rankings can be 
effectively used as a basis for identifying and ranking goals for use 
in the individual instructor professional growth program. 

The results indicated that student ratings of items in the 
three major element groups and of the overall instructor performance 
were done in a consistent and discriminating manner; the test of relia- 
bility was positive and high. Significant differences in the student 
overall ratings of instructors were found in comparison to those 
student elements of expected grade and present opinion of course. 
Also, significant differences in student overall ratings of instructors 
were found between female and male students and among students in 
courses in different instructional divisions of the colKge. Student 
ratings to only five items in course and instructor element groups were 
found to relate significantly to the student overall ratings of 
instructor. 

One recommendation was to continue use of student ratings to 
collect data about the instructor's performance. A second was to base 
appropriate adjustments to the student overall ratings of instructors 
on significant elements, gender of student, and college division in 
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which the course was offered. A third recommendation was that 
determination of the amount of adjustment to student ratings be mic'-; 
for each element, student demographic, and college course and program 
classification in which a significant biasing influence was found to 
exist at the college. A fourth recommendation was to use adjusted 
student ratings as a basis for identifying and ranking goals to be 
included in the individual instructor's professional growth program. 
Lastly, it was recommended that subsequent periodic studies be 
conducted to monitor relationships and effects on student ratings of 
elements and instructors as new programs emerge and composition of the 
student body changes. 
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CHAPTER 1 
INTRODUCTION 



This Major Applied Research Project (MARP) was conducted to 
determine the utility of student ratings of instructional elements for 
identifying and ranking instructor professional growth goals. This is 
especially important to the operation of an instructor growth program 
at institutions of higher education, especially at the Lakeshore 
Technical College (LTC). Chapter One includes the following ten 
sections: (1) Background and Significance, (2) Statement of the Prob- 
lem, (3) Purpose of the Study, (4) Major Elements, Considerations, and 
Research Questions, (5) Research Hypotheses, (6) Implications for the 
Improvement of Educational Practice, (7) Definition of Terms, 
(8) Limitations, (9) Assumptions, and (10) Delimitations. 

Background and Significance 

Vocational education began in Wisconsin in 1911. At that 
time, vocational agriculture and homemaking programs were offered. 
Secondary school programs were expanded to offer evening and other 
weekday instruction to farmers and homemakers to assist them in 
becoming more productive. 

During the period between 1911 and 1965, the University of 
Wisconsin developed an Extension Division for continuing education in a 
number of fields. At the same time, legislation was passed enabling 
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municipalities to organize and administer vocational education programs 
was passed. Statutes were also passed that allowed municipalities hav- 
ing populations over 10,000 to organize occupational education pro- 
grams, levy taxes to support these programs, and hire a director to 
administer these programs. Although the educational opportunities for 
job preparation were increased, a number of persons were still without 
these opportunities because they lived in the many municipalities that 
had populations of less than 10,000; and opportunities for them were 
not included in the legislation. In addition, there was a lack of 
standardization since each municipality was responsible for its own 
offerings. 

In 1965, the Wisconsin legislature passed legislation to 
create a uniform system to provide public vocational, technical, and 
adult education to all persons over the age of sixteen within the 
state. The educational offerings were to be financed primarily by 
local property taxes, student tuition and fees, and state aid based 
upon a ^Ci.mula established by the state legislature. McGown (196vS:l) 
concluded that the three primary motivating factors leading to the 
legislation were 

That the state's citizens deserved higher quality vocational, 
technical, and adult education programs that the district plan 
could best promote this quality and that the school system should 
becc.Tie truly statewide in scope. 

The Wisconsin Vocational, Technical and Adult Education (VTAE) 

system was developed to train postsecondary school persons (age sixteen 

and over) for the world of work. This mission, based on 1965 and 1971 

efforts, was restated in the Wisconsin statutes (1975, Chapter 38:1), 
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* • programs with specific orientation below the baccalaureate level 
• . (see Appendix A), The mission continues to be important for all 
vocational education agencies as Asche and Vogler (1980:16) stated, 
"The long-term goal of vocational education is to meet the manpower 
needs of the state and the nation. The needs have been construed to 
mean sufficient quantities of trained persons for the labor force," To 
accomplish this, legislation was passed in 1965 to organize the 
Wisconsin system of occupational education consisting of eighteen 
districts by 1970. Each resident over sixteen was thereby provided 
access to public postsecondary occupational education. 

State control and coordination was provided by the state voca- 
tional, technical and adult education (VTAE) board. The board hired a 
state director, a staff of educational consultants, and a support staff 
of fiscal consultants. This was in accordance with the recommendation 
made by HcGown (1968:45-46) to the Wisconsin Department of Administra- 
tion in which he emphasized that "districts must become well organized. 
The state office staff must provide the leadership to help the 
districts realize that this is the most critical challenge faced by the 
system," 

As a result of challenges, hearings, etc., the eighteen-dis— 
'.rict structure has been reduced to the present sixteen districts (sea 
nd^x B), One of these districts is the Lakeshore Vocational, 
Technical and Adult Education (VTAE) District, with its physical campus 
known as Lakeshore Technical College (see Appendix C), The name change 
from Lakeshore Technical Institute (LTI) to Lakeshore Technical College 
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(LTC) occurred in July 1987. The mission has been further defined by 
the district board in its North Central Self-Study report (Lakeshore 
Technical Institute, 1980:10) as "the preparation of an individual for 
initial or continued gainful employment." 

The Lakeshore VTAE District operates fifty-five less-than-one- 
year, one-year, and two-year vocational diploma programs and two-year 
associate degree programs. The following types of programs are also 
offered: 

1. Apprenticeship training programs. 

2. Adult and continuing education programs. 

3. Short-term programs. 

4. Seminar-type programs. 

5. Farm training programs. 

6. Consumer topics programs. 

7. Technical assistance programs. 

These programs are designed primarily to meet the manpower needs of the 
two-county area served. However, several programs are offered to meet 
regional and statewide needs. 

Approximately one hundred contract full-time and part-time 
(employed for eighteen to thirty-four hours per week) instructors are 
employed to provide instruction in various programs at LTC. Approxi- 
mately 450 call staff (employed for fewer than eighteen hours per week) 
are also employed as needed to teach in programs. All instructors must 
be certifiable when hired to teach courses in programs. Certification 
standards for instructors were developed by state VTAE staff and 
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approved by the VIAE board. Certification standards are included in 
the Wisconsin Administrative Code (see Appendix D). 

Effectiveness of the Wisconsin VIAE system in carrying out its 
mission has been due to its highly qualified technically proficient 
Instructors. Competence in the occupational area has been emphasized 
In selecting instructors. Teaching experience has been an advantage 
but not a requirement for a new instructor. Certification addressed 
this through a renewable provisional status (see Appendix D for 
details) while new instructors completed at least one course in each of 
six areas before receiving a five-year certification status. These 
areas include: 

1. Principles of VTAE. 

2. Course construction. 

3. Teaching methods. 

4. Educational psychology. 

5. Educational evaluation. 

6. Intergroup relations. 

Instructors in the Wisconsin VTAE system must maintain 
certification to teach. The five-year full certification status is 
renewable upon evidence that appropriate work experience and/or course 
work has been completed during the previous five-year period. Failure 
to maintain certification voids the contract between the district and 
the instructor. 

Instructors have no tenure in the Wisconsin VTAE system. 
However, labor laws involvinn bargaining units, continuing contracts. 
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due process, etc., are in effect. These ensure continuous employment 
after the probationary period has been completed except for specified 
causes such as low enrollment, unsatisfactory teaching performance, 
etc. 

The staff professional growth program for providing continuous 
quality instructional staff has been and continues to be the responsi- 
bility of the administrative staff of each individual district. In 
prior years, identification of professional growth goals was primarily 
the instructor's responsibility. Bases for identifying and ranking 
goals were the analysis results of evaluations of instructor perform- 
ance; e.g., self-evaluation, supervisor evaluation, and student 
ratings, with student ratings being most important. 

Prior to 1986, district procedures included little reference 
to an individual instructor professional growth program except through 
certification renewal activities or general in-service activities as 
long as the instructor demonstrated satisfactory instructional perform- 
ance. Primary indicators of satisfactory performance were (1) average 
or better student ratings and (2) lack of student complaints. 
Supervisor classroom observations were not normally included in 
instructor evaluations unless indications of concerns were received. 

Prior to 1986, student ratings at LTC were obtained on an 
Instrument containing fourteen items. Fach item had a Likert numerical 
scale, one to seven, for use as student choices. Responses from 
students in all vocational diploma and associate degree-level courses 
were obtained during 2ach semester and during summer school. The mean 
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for each of the fourteen items was calculated for each instructor, 
department, division, .and the college as a whole. The respective data 
were provided to the instructor and instructor's supervisor for use 
when discussing the instructor's proposed professional growth program. 

In 1984, the concern that student ratings of items on the 
existing instrument were not appropriate for professional growth deci- 
sions gained support of a number of instructional and management staff. 
A quality circle, a small group of employees that identify a problem 
and seek a solution (see Quality Circle, page 21), was organized to 
study the concern and develop recommendations for alleviating the 
concern. Circle members reviewed the concerns, specific items, and use 
of the results of tne student assessments, usually referred to as stu- 
dent ratings. A nuuber of instruments used for student ratings of 
instruction were reviewed in the process. 

After a year and a half of effort, a new instrument (see 
Appendix E) for student assessment of instruction was developed. This 
instrument contained thirty-one items, each having five choices of 
which the student was to select jne. Items were organized into three 
major groups: course elements, instructor elements, and student 
ele-nents. In addition, two" open-ended items were included for which 
students list two accolades for the course and/or instructor and two 
suggestions for course and/or instructor improvement. 

Prior to 1986, staff professional growth at LTC was primarily 
the instructor's responsibility. This was especially the case with 
established instructors who had been judged to be good instructors by 
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division supervisors and students. Results of student ratings were 
used in setting goals for instructor growth, only if student ratings 
were low. Instructors also benefited from a district-supported pro- 
fessional growth program which included such items such as planned 
in-service programs, tuition reimbursement for college courses, and 
payment of expenses for attendance at seminars and workshops. 

The LTC Instructional Services Division is currently involved 
in implementing a continuous individual professional growth program 
known as the LIFE program. The LIFE program, an acronym for Life-Long 
Investment For Excellence, was developed during .1985-86 by four 
managers in the LTC Instructional Services Division. This program was 
developed to remove the anomaly that while one LTC board policy 
required an annual instructor evaluation, it did not require develop- 
ment of a professional growth plan for that instructor. 

The LTC instructional administrators firmly believe that, 
depending upon the extent to which each becomes a reality, the 
following are important to the success of the LIFE professional growth 
program: 

1. Professional growth of the individual instructor should be 
the focus. 

2. Instructor's professional growth goals should be identi- 
fied and ranked using a process involving both instructor and 
administrator. 



3. Identification and ranking of instructor's professional 
growth goals should be based on data. While primary emphasis should be 
placed on data from student ratings of instruction, self and adminis- 
trator assessments should also be included. 

4. Activities and resources required to achieve the selected 
professional growth goals should be identified. 

5. Commitments of resources by the college, instructor, and 
administrator are essential if goals of individual professional growth 
plans are to be achieved. 

6. Data should be collected to provide an assessment of 
progress made in achieving the instructor's professional growth goals 
and to provide a basis for identifying and ranking new goals. 

7. The steps included in the professional growth program 
should be repeated in a continuous spiral toward a sel f -actual i zed 
process. In this process, the instructor's level of responsibility for 
identifying and ranking goals, activities, and resources increases 
while administrator involvement gradually decreases (see Appendix F). 

A major component of the LIFE professional growth program is 
the data on which decisions about the instructor's growth plan will be 
based. - These data are to be largely derived from student ratings of 
instruction. There are numerous course, instructor, and student 
elements (such as course organization, instructor enthusiasm, and 
«:tudent opinions); student demographics (such as age); college course 
and program classifications (such as function) that may affect student 
ratings either positively or negatively. Therefore, it is important 
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that the influence of each of these elements on student ratings be 
known so that appropriate adjustments to these ratings can be made. 
The adjustments to student ratings will establish comparable values for 
the ratings. These adjustments and considerations should promote staff 
acceptance of the LIFE professional growth program. This program 
directly affects the individual professional growth of more than 100 
LTC full- and part-time contract instructors and approximately 450 call 
staff instructors. 

A description of steps included in LIFE indicates the use of 
this study's results. First, student ratings for each item indicating 
a course, instructor, or student element are obtained, and the rating 
mean calculated. Second, an appropriate adjustment is made to each 
item rating mean depending on the extent of biasing influence of the 
course, instructor, or student element; student demographic; or college 
course or program classification. Third, a comparison between the 
adjusted student rating mean and the desired student rating mean for 
each item is made. Those items for which a considerable difference 
between the two means exists are noted. Fourth, noted items are ranked 
according to the relative relationship between each item's mean and 
students' overall ratings of instructor mean. Fifth, each listed item 
is translated into a corresponding course, instructor, or student 
element. Sixth, instructor profossional growth goals are developed and 
ranked in accordance with ranked elemen s. Results of this study have 
a direct impact on the outcomes of steps two and four. 
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statement of the Problem 

LTC instructional staff members do not currently know the 
biasing effect of particular course, instructor, or student elements; 
student demographics; or college course or program classifications on 
student ratings of instructor performance. Staff members also do not 
have knowledge of relationships existing between student ratings of 
particular elements and student overall ratings of instructors. 
Without such knowledge, appropriate adjustments to the student ratings 
cannot be iiiade; and data upon which goals are developed and ranked are 
less accurate than desirable. 

Purpose of the Study 

The purpose of this study was to refine a part of the instruc- 
tional assessme'-'> procedures at LTC. This was accomplished by obtain- 
ing and analyzing data for course, instructor, and student elements to 
determine their relative relationships to student overall ratings of 
instructors. Equally important, this study also sought to determine 
whether there were biasing effects of particular course, instructor, 
and student elements; student demographics; and college course and 
program classifications on student overall ratings of instructors. 
This determination is important in order to make necessary adjustments 
to student ratings and their use. Such adjustments are necessary to 
ensure compatibility in interpreting student ratings and ranking 
instructional concerns. After the instructional assessment procedure 
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has been refined, it seems reasonable that instructors will be more 
accepting of it as a useful tool in their professional g.-:wth program. 

Major Issues and Research Questions 

Several major issues were addressed in this study. The first 
issue is whether an evaluation of an instructor's teaching can be made. 
Aubrecht (1979:1) questions any evaluation on the basis that "no one 
has found . perfectly reliable and relevant (and therefore valid) 
measures of teacher effectiveness." Johnson (1984:91) indicates, "The 
process of faculty evaluation is complex and. open to debate." Cashin 
and Perrin (1983:595) agreed that "there is no generally accepted 
behavioral domain for 'effective teaching'" but contended that this 
should not prevent the use of evaluations for use in faculty develop- 
ment. Centra (1977:50) stated that "unfortunately," many instructors 
believe they are best able to judge their own needs and do not need or 
want to include evaluation by others in the process of determining 
professional growth needs. Braskamp, Brandenburg, and Ory (1984:19) 
acknowledge that teachers may believe this but contend that a major 
purpose of evaluation is "to help faculty to examine their teaching 
performance to help them improve." 

Another issue comes after whether or not evaluation is useful. 
That issue is who is in a position to conduct a valid evaluation. Each 
(self, peer, supervisor, and student) has advantages and limitations. 
McKeachie (1983:37) indicates the issue in "Who is competent to judge 
the relative values of the many different kinds of teachers who make up 
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the faculty?" The many different kinds of situations in which 
Instruction occurs also adds to the complexity of the issue. 

Another issue relates to one particular type of evaluator, 
that being the student. The issue is the value of student ratings. 
Everett (1981:327) indicates the ext<insivene<;s of the issue in "Casual 
surveys of the literature and practices suggest considerable contro- 
versy over student evaluations of teaching." Hunter (1982:3) concurred 
in "The issi'e of the ef /ectiveness of student ratings in improving 
instruction is not closed." 

Finally, a major issue is the validity and reliability of 
student ratings in identifying good instruction and not being biased by 
elements that the instructor may or may not be able to control. 
Costin, Greenough, and Menges (1971:511) stated that many educators 
question the "inference ths.t instructors who obtain high student 
ratings are actually better teachers." They contended that many 
instructors believed that "student ratings are unreliable, that they 
favor the entertainer over the instructor who gets his message across." 
However, they (1971:512) also expressed the view of those on the other 
side of the issue in, "Proponents of student ratings have held that 
[ratings] ... are testable." 

There will continue to exist those who believe that students 
can be manipulated so that the ratings obtained will be biased. A 
number of course, instructor, and student elements; student demo- 
graphics; and college course and program classifications included in 
this issue were also included in the study. Those included in this 
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study were selected by the LIFE program developers as important to the 
growth program of the instructor. 

The elements included in this issue and established for use in 
this study were: 

1- Course--items concerning assignments, evaluations, 
function, information, organizational division, and level, 

2, Instructor—items concerning application of knowledge and 
skills, enthusiasm, interest in student, instructional techniques, 
organization of classes, respect for student, and subject knowledge, 

3, Student--items concerning difficulty with course, expected 
grade, initial opinion of course, perceived out-of-class preparation, 
and present opinion of course. 

Considerations of student demographics of age and gender were 
included. College course and program classifications of function, 
level, and instructional division were elso included, LIFE program 
developers considered these important because of the issue about the 
potentially positive or negative impact on the student overall ratings 
of instructors. These demographics and classifications were included 
because of their potential biasing influence even though the instructor 
had little or no control over them. 

The research questions developed for this study were: 
1. Is there a relationship between student overall ratings of 
in:tructors and student ratings of items in the three major element 
groups (course, instructor, and student) in the assessment of 
instruction at LTC? If so, what is the nature of the relationship? 
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2- Is there a relationship between student overall ratings of 
instructors and student ratings of particular items within each of the 
three major element groups, student demographics, or place of the 
course in college academic programming? 

Answers to these questions are needed to make adjustments to 
the item ratings to compensate for specific course, instructor, and 
student influences. They are used to help identify goals. They are 
also used to determine the priorities of goals and activities for 
inclusion in the individual instructor's professional growth program. 

Research Hypotheses 

The research hypotheses developed for this study were: 

1. Differences in student overall ratings of instructors are 
related to different ratings that students give to the elements, as an 
aggregated measure on the rating instrument, that pertain to the 
(a) course, (b) instructor, and (c) student. The aggregated measure 
for each element is the composite of the separate items pertaining to 
the indicated elements. 

2. Differences in the student overall ratings of instructors 
are related to the ratings students give to each of the items on the 
rating instrument that pertain to the first element in research ques- 
tion one (the course) . 

3. Differences in the student overall ratings of instructors 
are related to the ratings students give to each of the items on the 
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rating instrument that pertain to the second element in research 
question one (the instructor). 

4. Differences in ihe student overall ratings of instructors 
are related to the ratings s jdents give to each of the items on the 
rating instrument that pertain to the third element in research ques- 
tion one (the student). 

5. There is a significant difference between the overall 
ratings of instructors by students in courses having one function and 
by students in courses having another. 

6. There is a significant difference between the overall 
ratings of instructors by students in one course level and by students 
in another. 

7. There is a significant difference between the overall 
ratings of instructors by students in one age group and by students in 
another. 

8. There is a significant difference between the overall 
ratings of instructors by students of one gender and by those of the 
other. 

9. There is a significant difference between the overall 
ratings of instructors "ly students from one division and by students 
from another. 

10. There is a significant difference between the overall 
ratings of instructors by students expecting one grade in the course 
and by students expecting another. 
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11. There is a significant difference between the overall 
ratings of instructors by students exerting one level of out-of -class 
preparation and by students exerting another level. 

12. There is a significant difference between the overall 
ratings of instructors by students feeling the course had one level of 
difficulty and by students feeling the course had another level. 

Implications for the Improvement of Educational Practice 

The intent of this MARP was to improve procedures used in 
establishing a reliable and valid data base using student ratings of 
instructors that would be useful for developing goals for the instruc- 
tor's professional growth program. The procedures used in this study 
have potential value to educators in colleges using student ratings of 
instructors as a basis for making decisions about instructor profes- 
sional growth. The procedures used in this study could be helpful to 
those attempting to determine the adequacy of using raw student ratings 
in making interpretations and comparing teaching performances of 
instructors. The procedures could also be useful to those educators 
attempting to develop procedures for determining the biasing influences 
of course, instructor, and student elements; student demographics; 
and/ or col 1 ege course and program cl assi f i cat i ons . Fi nal ly , the 
procedures used in this study could be used by those educators attempt- 
ing to develop procedures for necessary adjustments to raw student 
ratings in courses that are atypical in terms of elements, 
demographics, and/or classifications. By incorporating these 
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procedures into their own instructor growth programs, in which 
decisions are based on. student ratings, educational leaders could gain 
increased instructor acceptance and participation. 

Definition of Terms 

A number of terms are used in this report in a special way. 
They are basic to the project. The definition of each is as follows: 

Administrator Evaluation - An assessment of instructor 
strengths and weaknesses conducted by an associate administrator for 
the purpose of providing data for decision making regarding instructor 
growth goals and activities. Prior to reorganization in July 1987, 
this activity was conducted at the supervisor level, which is currently 
one level below the associate administrator level. 

Assessment Rating - A point response on a one to five scale by 
a student to an item on the Student Assessment of Instruction form. It 
is a judgment of the agreement of the response choice to the student's 
feeling regarding an item. The word "assessment" has been substituted 
for "evaluation" to indicate a paradigm shift i.i the use of student 
ratings of instruction at LTC from "evaluation" associated with an end 
result to "assessment" associated with strengths (talents) and 
weaknesses for decisions regarding growth. 

Associate Administrator - A manager of a division which offers 
a number of occupational programs and services at Lakeshore Technical 
College. The associate administrator, also referred to as administra- 
tor, is at the fourth level from the top in LTC's organizational 
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hierarchy. One associate administrator responsibility is to assist the 
instructor in determining the goals for the individual professional 
growth plan for the following year. 

Contract Instructor - An instructor employed eighteen or more 
hours per week for the purpose of providing instruction in an 
occupational program. 

Course Function - A course classification system developed by 
the Wisconsin Board of VTAE staff to categorize intent of a course in 
an associat3 degree or vocational diploma program. The four "func- 
tions" are (1) occupational specific, (2) occupational supportive, 
(3) general education, and (4) elective. Ratings from students in 
occupational -specific and general education courses were included in 
this study. 

District - One of sixteen geographic areas defined by 
Wisconsin statute to provide postsecondary occupational and adult basic 
education at the less than baccalaureate level. District is short for 
vocational, technical and adult education district. Each district has 
at least one physical plant known as a college. 

Division - The occupational grouping of which the occupational 
program is a part. It is also referred to as program area. Examples 
include business and marketing, health occupations, home economics, and 
trade and industry. 

Division Assistant - A support staff person at LTC that 
assists an associate administrator in the operations of a division. 
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One responsibility of the division assistant is to administer Student 
Assessment of Instruction instruments to students. 

Full -Time Contract Instructor - An instructor defined by the 
negotiated agreement between the Lakeshore VIAE District Board and the 
Lakeshore Education Association providing a minimum equivalent of 
eighteen hours of instruction per week during the semester. 

Full -Time Occupational Program - A postsecondary educational 
program relating to a cluster of jobs or a specific job having thirteen 
to eighteen credits of courses offered each semester. Total number of 
credits in an associate degree program ranges froni sixty-four to 
seventy-two credits and from twenty-six to sixty-five credits in a 
vocational diploma program. Both part-time and full-time students are 
enrolled in full-time programs. 

Instructor Professional Growth - A process whereby additional 
knowledge and skills are gained to better instruct in a variety of 
settings, to feel comfortable with students, and remain relevant with 
regard to the new technologies in the workplace and in instructional 
del ivery. 

Instructor Growth Program - A planned grouping of goals and 
activities for improvement of instructional skills in which goals are 
developed, activities selected, commitments obtained, and faedback 
received about instructor's progress in achieving these goals. 

Lifelong Investment for Excellence (LIFE) - A staff growth 
program developed by instructional managers at Lakeshore Technical 
College. LIFE emphasizes use of student ratings of instruction and 
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data analysis and depends on instructor and division administrator 
interaction for identifying and ranking goals for individual instructor 
professional growth. 

Program Level - An indication of its hands-on vs. theoretical 
proportion. The VIAE system has designated two levels, technical lev9l 
with a greater theoretical emphasis and vocational level with a greater 
hands-on emphasis. 

Program Sequence File - A computer file that includes all 
occupational programs at LTC. Courses are listed in numeric order with 
the lowest number first (e.g., 091-100, Basic Riding, is listed before 
801-15', Communication Skills I) for each term of each program. 

Quality Circle - A "small group of employees who do similar 
work; volui.tarily meet regularly to identify and analyze causes of 
problems; recommend their solutions to management; and where possible, 
implement solutions" (Ladwig, 1983:12). 

Self -Evaluation - An instructor's self- assessment of strengths 
and weaknesses in Instructional delivery to provide data for decisions 
regarding instructor professional growth goals and activities. 

Student Assessment Rating - The student's response to each of 
thirty-one items included on the Student Assessment of Instruction 
instrument. Ratings provide data for decisions regarding instructor 
professional growth goals and activities. 

Wisconsin Vocational. Technical and Adult Ed jcation (VIAE') 
System - A public postsecondary system of education in Wisconsin estab- 
lished for the purpose of offering occupational programs and services 
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below the baccalaureate level. Sixteen districts having geographic 
boundaries were established so all persons in Wisconsin h?.ve access to 
occupational programs and services. Each district has its own district 
board for governance and operates under the guidance and direction of a 
state board,. Each district is financed through contract funds, grants, 
property taxes, state aids, and student tuition. 

Limitations of the Study 

The study had a number of limitations. These included: 

1- Only students in attendance when the Student Assessment of 

Instruction instrument was administered were included in the study. 

Including ratings from students absent on that day could have resulted 

in different means for student ratings, 

2, The data collected were limited to accuracy and honesty of 
assessments made by students. Students were counseled about the 
importance of accuracy and honesty in their ratings. However, time and 
resources available to complete this project precluded efforts to 
verify either. Therefore, the study results must be treated 
cautiously, 

3, The assessments were limited to students enrolled in full- 
time, one- or two-year programs at the technical or vocational program 
levels. Students in part-time programs or other types of educational 
programs could have responded differently, 

4, Student assessments were only collected in courses desig- 
nated as first or second semester courses in an occupational program 
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offered during the fall semester of the 1986-87 school year. Students 
in the second year of a program could have responded differently. 

5. Students in occupational and general education function 
courses were similar but not necessarily identical. Therefore, there 
was not a one-to-one relationship between students that completed the 
Student Assessment of Instruction instrument in general education 
courses and students completing the instrument in the occupational- 
specific courses. 

6. Students enrolled in the first general education course 
and the first occupational -specific course listed in the program 
sequence file for selected programs (or a substitute course having the 
same students) were included in the study. Students in these courses 
may not have been representative of the population; and therefore, the 
ratings may not have been representative. 

Assumptions 

This study was buiU upon a number of assumptions. They 
included: 

1. Elements included on the Student Assessment of Instruction 
instrument were elements that potentially would have the greatest 
influence on overall ratings of instructors and be most important in 
providing data for use in identifying and ranking instructor profes- 
sional growth goals. 

2. Administration information and instructions given students 
for completing the instrument were similar in each course section. 
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Written instructions were provided to administrative assistants 
administering the assessment instrument during a preselected class 
period. 

3. Students not present and, therefore, not responding to the 
assessment instrument would have responded in the same pattern as those 
that did respond* 

4. Interpretations of each of the response choices for each 
item on the Student Assessment of Instruction instrument by students in 
each group was similar. The students responded using a common 
subjective metric interpretation of the response choices. 

Del imitations 

The study had several delimitations. They included the 
following: 

1. Data and implications were developed for use as a basis 
for identifing and ranking individual instructor professional growth 
goals only. No attempt was made to provide data or implications for 
use in evaluation for other purposes or other research. 

2. The study was limited to students enrolled in courses in 
occupational programs offered at only one institution; that is, 
Lakeshore Technical College. 

3. The scope of the study; i.e., number of courses and stu- 
dents involved, was limited to computer capacity and resources avail- 
able to the investigator. 



CHAPTER 2 
REVIEW OF RELATED LITERATURE 



This chapter reports on literature related to the study of use 
of student ratings of courses and instructors in higher education as 
well as their utility in identifying individual instructor's profes- 
sional growth goals. Chapter Two includes the following twelve 
sections: (1) Instructor Professional Growth Definition and Purpose, 
(2) Need for Professional Growth in Postsecondary Educational Institu- 
tions, (3) Effectiveness of Professional Growth Programs, (4) Content 
of Professional Growth Programs, (5) Importance and Use of Evaluation 
in Professional Growth, (6) Importance of Student Ratings of Courses 
and Instructors in Professional Growth, (7) Reliability and Validity of 
Student Ratings of Courses and Instructors, (8) Importance of Course, 
Instructor, and Student Elements; Student Demographics and Course and 
Program Classifications in a Student Rating of Course and Instructor 
Instrument, (9) Correlations and Influence of Course, Instructor, and 
Student Elements; Student Demographics; and Course and Program 
Classifications on Student Ratings, (10) Administration of Student 
Rating Instruments, (11) Use of Student Ratings in Identifying Indi- 
vidual Professional Growth Goals, and (12) Summary. 
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Instructor Professional Growth: 
Definition and Purpose 

Postsecondary institutions must develop an effective profes- 
sional growth program to enable instructors to better provide instruc- 
tion that is relevant and acceptable to students. Importance of 
professional growth programs is indicated by Koerin {i980:40), "Faculty 
development [growth] programs have been seen as offering a means by 
which an institution might stimulate its faculty and improve teaching 
capabilities." Hunter (1982:7) emphasized this further, "The capacity 
for new xperience for renewal and growth is essential to teaching 
effectiveness." 

Professional growth programs for instructors are projected to 
increase in number and will have greater acceptance. Altshuler 
(1985:60) predicted the value of such programs based on need for 
quality education by stating, "As the promotion of teaching excellence 
becomes a priority, more people besides teachers will value it [staff 
growth] . . . We need to develop more teaching excellence." Bender 
and Lukenbill (1984:18) used others to support a professional growth 
program aimed at individual change: "In their book. In Search of 
Excellence . Peters and Waterman report that a pervasive theme in excel- 
lent corporations was the tough-minded respect for the individual." 
They advocate that the same emphasis be placed on individuals to make 
improvement needed to enable colleges to fulfill their teaching mission 
as is expended in Industry to increase productivity. Altschuler 
(1985:60) reinforced this, "We need to emphasize what teaching 



27 



means. ... We need to develop more teaching excellence." In the 
past, institutions have been concerned with providing education and 
training for large numbers of students. If two-year colleges are to 
continue to be viable providers, "quantity education focused on 
increased access must give way to quality education concerned with 
student achievement" (Richardson and Rhodes: 1985:296) . Faculty growth 
must be a meaningful endeavor, not just a response to rapidly declining 
enrollments aif.' resources. As Berquist and Phillips (1975:11) sug- 
gested, "Faculty development [growth] is more than merely a response to 
crises and retrenchment for it fundamentally offers new and consider- 
ably more complex paradigm in higher education than held in the past." 

A variety of definitions and terms are used for the term 
"professional growth." Centra (1976:5) referred to the term in these 
words, "The term faculty development (at times simply development) is 
used to encompa*, the broad range J activities institutions use to 
renew or assist faculty in their varied roles." Taff (1975:4) used the 
term "faculty development" as "it allowed the incl-jsion of activities 
related to the effect of development of faculty members." He also 
indicated that the term "facu.ty" was important so that "those directed 
toward i '^roved teaching behavior" were directing their efforts in the 
same direction--toward the instructors' activities involved in improved 
student learning. 

• Harrell (1980: HI) suggested *:hat faculty growth "may be 
defined as enhancir.3 the talents, expandi'-g the interests, improving 
the competenc), and otherwise facilitating the professional and 
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personal growth particularly in their roles in instruction." He 
further defined a growth program in terms of objectives. He listed 
three principal objectives for a faculty growth program as: 

1. To facilitate the professional and personal growth of 
faculty members in their role a: instructors. 

2. To improve instructional effectiveness. 

3. To encourage professional growth in accordance with its 
[the institution's] mission and goals. 

According to Griffin (1983:2), professional growth is defined as "a 
purposeful endeavor. It is a deliberate activity generally undertaken 
with specific purposes or goals in miiid. The changes ... can usually 
be well defined." 

Professional growth should be thought of as a process in which 
there is improvement of the faculty member's performance as an 
instructor. Hammons, Wallace, and Watts (1978:1) emphasized that 
"development can be thought of as synonymous with improvement-- 
improvement measured in term:> of increased efficiency (dointj things 
better) and effectiveness (doinr the right thing better)." 

It is proposed that several types of changes result from 
participation in instructor growth activities. To be more effective as 
an instructor, several types of changes may be required. After listing 
a number of changes, Tom (1986:12) summarized them as "Changes in 
Teacher Attitudes ---> Different Classroom Practices ---> Improved 
Student Learning." 
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Continued professional growth is necessary to enable 
instructors to continue to teach effectively as conditions {e,g,, 
knowledge, students, and technology) change. Just as initial training 
was necessary to become an effective instructor, professional growth is 
necessary to remain instructionally effective. Bishop (1976:1) made an 
interesting analogy as he stated, "Staff development . . . activities 
are the career counterparts of preservice education. And as such, thpy 
provide for change, renewal, quality education, and professional 
competence," 

An important consideration in the development of professional 
growth programs has been acceptance by staff at institutions so that 
such programs can make a difference. If teaching is considered a 
science, changes as a result of participation in professional growth 
activities can also result in more student learning. After reviewing a 
number of studies and also conducting a study in this area, Georgea 
Sparks (1986:224) concluded that this "indicates teachers can under 
certain conditions and in a relatively short period make desirable 
changes in thei' teaching," This conclusion is a very significant 
finding for the value of in-service education, 

Guskey (1986:5-6) provided a brief history of professional 
g jwth programs. He stated that teacher institutes, as they were known 
then, began in the late nineteenth century to provide opportunity for 
instructors. They were not very effective because they wera "charac- 
terized primarily by disorder, conflict, and criticism," Instructors 
were provided wilh resources to do whatever they wished. They did not 
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necessarily define goals or select activities that would result in more 
effective student learning. 

This type of loosely organized professional growth program 
seems to have characterized most of the program* until the mid-1970s. 
After a reviaw of a number of growth programs, Davey (1985:2) summa- 
rized them by stating that "since the mid-1970s, many -alleges and 
universities have implemented faculty development programs aimed at 
improving instructional practices." Emphasis on programs aimed at 
improving an individual instructor's teaching competence is, thus, 
relatively recent. 

Centra (1976:?) has also rorked with professional growth over 
the years in his capacity with Educational Testing Service and has 
recently seen a shift in emphasis in growth programs. He identified 
two reasons for the increased emphasis in postsecondary institutions. 
The first .was related to retrenchment and its consequence as "there has 
been a decrease in faculty mobility due to declining rate of growth 
. . . colleges can n longer depend on new staff to keep them vital; 
nor can teachers broaden perspectives by changing jobs." The second 
was related to the public's demand for quality as he reminds us that 
"another reason for tne recent emphasis on faculty development and 
instructional improvement is the general disenchantment . . .with the 
quality of instruction." These rea.«;ons were reiterated by Centra and 
others even more recently (Centra, 1980:2; Eble, 1983:122; and Manns, 
1985:269). 



31 



Professional growth programs are also becoming more involved. 
Most original programs provided resources for the instructor to work on 
activities related to goals set by the individual. Many recent 
programs have been organized and have staff to assist an instructor in 
developing goals and completing activities related to teacher improve- 
ment. Goldschmid (1978:233) studied a number of recently organized 
staff growth programs and concluded that "a number of universities have 
establ ished staff development programs, often organized by speci2l 
units. . . . These units are supposed to provide the impetus for 
teaching improvement efforts." Guskey (1986:5) further emphasized the 
purpose of professional growth as being to "alter the professional 
pract i ces , bel i ef s , and understandi ngs of school persons . " Average 
ages of college student bodies are increasing and as instruction is 
being conducted in-plant for the purposes of upgrading and retraining 
the work force, many instructors are becoming uncomfortable when first 
faced with this type of instruction. Koerin (1980:43) observed that 
"many college professors may feel ill prepared to meet the educational 
needs of the new group*- of college students." She further indicated 
that these were student groups "whose learning styles or life styles 
may require innovation in course design and in teaching styles." 

Hammons, Smith-Wallace, and Watts (1978:4) provided additional 
support for Koerin's remarks regarding ''student clientele," but also 
loc;:ed at the situation in a broader scope and included recent 
acceleration of the developme^nt of a technology of instruction and its 
effect on "redefining the teaching role." Cross (1977:11) warned 
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academic leaders that instructors could make necessary transitions as 
the new student clientele came, "but it may require a painful transi- 
tion period as faculty reoriented to new skills and satisfactions." 
Professional growth programs were, thus, promoted to ease the 
transition period through the use of guidance and support. 

Instructors are also remaining longer than previously at the 
same college. At the same time, lower enrollments have re«L'Hed in 
fewer new instructors being hired. Consequently, that previous source 
of new ideas and challenges, newly hired instructors, has been almost 
eliminated. As a result, postsecondary institutions have had to 
provide challenges and staff growth activities to compensate for this 
loss of regereration which is resulting from lack of new instructors. 

Hendrickson (1982:341) emphasized thi'. as an issue to be 
addressed in ". . . many faculty members need stimulation and renewal 
if they are to maintain their interest in teaching effectiveness 
throughout their longer careers." Bender and Lukenbill (1984:18) 
explained why community colleges can no longer depend on the individual 
faculty member to adapt to new situations or to improve instruction 
alone. They defended the need for an organized professional develop- 
ment approach because "human nature is a reality. People gravitate to 
the status quo; hence, policies [regarding professional growth] must be 
followed by action." 

Licata (1986:1) also addressed this issue as she reviewed' 
impacts of budget restraints, steady state reallocations, declining 
enrollments, and retention problems as projected by educational 
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planners for the next decade and concluded, "These factors are further 
compounded by the fact that the absence of job mobility and shortened 
span of the career ladder have conspired to produce a feeling among 
some raculty of being stuck." Miller and Ratcliff (1986:316) addressed 
professional growth for the community college staff a.^ well. They 
confirmed the importance of professional growth, "especially during 
periods of low staff turnover and limited financial resources.*' 

In the past, instructors have not participated in staff growth 
programs as they grew older. A lifetime certification or license is no 
longer issued in Wisconsin at age fifty-five, so the instructor must 
continue to be involved in professional growth activities throughout a 
professional career. This has come about because new evidence indi- 
cates that faculty members are able to change their instructional 
presentations, even at older ages. 

Adult learning research indicates that adults can learn, and 
they can change given proper stimuli and responses. Sprinthall and 
Theis-Sprinthall (1983:23) reported on studies cond-cted by Baltes and 
Schaie. Baltes J<nd Schaie (no date) found that the "so-called decline 
in IQ was a myth." They also found that in "important areas of problem 
solving, generalization, and concept formation, there was no decline 
until after retirement age." Only in such areas of "vision and audi- 
tory perception and short-term memorization" did they find a decline 
with age. Sprinthall and Theis-Sprinthall then suggested on this basis 
that a professional growth program can be effective because of the 
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"concept of plasticity; that is, adults can learn new abilities and 
improve old ones throughout their teaching career." 

Need for Professional Growth in Postsecondary 
Educational Instituti *- is 

At the two-year postsecondary level, goals of education are 
related to knowledge for one's own interest and preparation for further 
education or job entry. Therefore, teaching approaches need to be 
student-centered. It is important that the instructor maintain the 
enthusiasm and continue in efforts to "make the course relevant" 
(Wilson, et al., 1975:20). 

Professional growth programs have recently taken a different 
direction. Originally programs were for the personal improvement of 
the faculty member. There was little or no regard for their impact on 
student learning. Lanier and Glassberg (1981:24) summarized this 
recent change in the statement: 

Recognition of the dynamics and unique qualities of teaching 
as well as the interdependent nature cf the teaching and learning 
ptocess led to the recognition in the last two decades of the 
need to study both these human activities (i.e., teaching and 
learning) competently and intentively. 

Professional growth program priority in institutions of higher 

education must be high as well. High priority for development of 

curriculum and alternate delivery mechanisms is good but will not be 

successful unless there is a high priority on professional growth as 

well. Rouche and Baker (1985:20) compared resources provided for- 

design and delivery of instruction with those provided for instructor 
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professional growth- They agreed with educational peers thac efforts 
"may indeed provide a better vehicle" but warned "that little lasting 
excellence will occur without a focus and commitment to excellence 
through the perfomance of human beings within the vehicle." 

There might be some concern that faculty members would not 
participate in a professional growth program directed toward the 
improvement of the individual. Studies conducted or cited by Guskey 
(1985:6); McLaughlin and Marsh (1978:75); Pellino, Boberg, and 
O'Connell (1981:13); and Wilson, et al • (1975:20) have resulted in 
evidence that was conclusive--community college instructors do care 
about teaching. Therefore, they are likely to participate in a program 
leading to improvement of their teaching skills so more effective 
student learning occurs. Impetus for participation in professional 
growth programs is summarized by John Rouche (1985:preface) as he 
proclaimed, "the ultimate winner or loser in our struggle for 
excellence is our students when we are teaching." Ross and Solumon 
(1985:5) concurred as they state^, "Simple deduction tells us that 
student: will be better educated if they are better taught." 

Cross and Small (1979:218) conducted a study in which they 
found little difference in effectiveness of growth programs incorporat- 
ing student ratings between nontenured and tr^'^ured faculty. Results 
from both groups indicated that using sx'jd^nt ratings leads to 
increased effectiveness of the growth program,?. Moderately increased 
instructional effectivenes: was reported by 33 percent of the 
nontenured and 44 percent of the tenured faculty. This also 
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contradicts a commonly held opinion that professional growth programs 
are important only for new instructors, not experienced ones. 

Effectiveness of Professional Growth Programs 

A number of factors influencing effectiveness of professional 
growth programs were identified. Goals and activities being identified 
and implemented for improvement of the individual was most often listed 
as being most important (Jones and Hayes, 1980:3; Lawrence, 1980:6; 
Pellino, Boberg, and O'Connell, 1981:17; Rodriguez and Johnstone, 
li,36,-95; Sprinthall and Theis-Sprinthall , 1983:24-25; St^ggins and 
Bridgeford, 1984:29). Importance of this factor appears to be rooted 
in the premise, "effectively changing the behavior of another person 
requires the enlisting of the cooperation of that person" (Stiggins and 
Bridgeford, 1984:30). 

Another explanation suggested is there is such a variance of 
talents and skills that no single program cin meet the needs of each 
individual. Even though perceptions of their own needs may not corres- 
pond with actual areas in which they lack knowledge or skill, instruc- 
tors must be able to include these needs at least in part to retain a 
feeling of ownership in the program. "Significant accomplishments have 
been achieved," concluded Beard and Hartley (1984:16) after reviewing a 
number of successful growth programs to determine why they were 
considered to be a success. ' "The key to the success of these units 
seems to be the willingness to assist the staff with che problems they 
bring." 
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Another explanation found for designing programs for 
individuals was that only individual needs are considered, thus giving 
focus to the program. PelUno, Boberg, and O'Connell (1981:17) empha- 
sized that needs of instructors differ and that "care should be taken 
to design programs capable of responding to individual differences that 
are dictated by differences in academic discipline, predilection for 
work activity, and career stage." This also ensures more instructor 
cooperation because as Eble (1983:134) observed, "Most people resist 
being taught what they already know .... The patient must acknowl- 
edge a need for treatment if the treatment is to be effective." He 
used this analogy to illustrate the importance of readiness of an 
instructor to participate in a growth program based on needs of the 
individual . 

Organizing the profession?l growth program using definite 
goals for the individual as a guide was found to be the second most 
important factor. Knowledge and skills resulting particularly in 
improvement of student-learning outcomes are to be emphasized. Harrell 
(1980:2) emphasized that the program needs to "create a constructive 
environment within which the instructional process can be fully 
explained in the interest of student success." Faculty members also 
participated in activities that lead to a desired end. Guskey (1985:5) 
acknowledged this as he reviewed a number of successful programs and 
agreed with Griffin (1983:2) that "programs vary greatly in context and 
format, yet they generally share a coiwnon purpose . . . designed to 
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alter the professional practices, beliefs, and understanding . . . 
toward an articulated end." 

A third factor indicated was the inclusion of elements that 
are attainable and relate to instructor satisfaction. Guskey and 
Easton (1983:272-273) advised educational leader's to include "primarily 
alterable characteristics and certainly [those] within the capabilities 
of most community college staff." They further suggest that programs 
should "encourage and motivate teachers to adopt practices and behav- 
iors such as these." They include a listing of behaviors considered to 
be valuable and effective in providing instruction and learning. 

A fourth factor found in successful programs was feedback. 
Guskey (1986:10) insisted that faculty members need to know how well 
they are progressing; therefore, "continued support and follow-up after 
the initial training" should be provided. This idea was proposed 
previously ly Braskamp, Brandenburg, and Ory (1984:20) and Cohen 
(1980:322), based on their belief that some instructional changes 
(attitudes, knowledge, and skills) were considered "long-range 
outcomes." Cohen (1980:322) classified the efforts needed to improve 
instmtion in two catt^^ories: "(1) . . . improvement of general 
teaching abilities in the instructor over time and (2) within-class 
improvement and instructional effectiveness evidenced over the course 
of a semester." 

A fifth factor identified in successful growth programs was 
incorporation of another person to assist in the growth process. This 
person could be a peer, a manager, or a staff development specialist. 
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Garman (1982:49), McKeachie (1983:39), and McLean (1986:5) emphasized 
that change is much more likely to occur if the instructor receives 
consultation as well. The consultant can provide one or more of the 
following services: 

L Guiding the faculty member in the direction of the model, 

Z. Interpreting and communicating data. 

3. Assisting in identifying goals and activities. 

4. Calling attention to information the instructor may not 

not^'ce. 

5. Acting as a resource person. 

6. Giving encouragement. 

7. Suggesting alternatives. 

The sixth factor found was that programs need to address 
instructor concerns and interests. Numerous staff growth programs have 
been organized on college and university campuses focusing almost 
excl usi vely on i mprovi ng i nstruct i on . As 81 ackburn and Lawrence 
(1986:284) studied a number of these programs, they concluded that many 
professional growth programs "seem to have low impact and did not 
address issues that faculty gave higher priority to; namely, profes- 
sional and personal growth." Therefore, these programs need to be 
redirected toward the areas the faculty members feel strongly about and 
are ready to devote their energies toward. 

Management staff at Lakeshore Technical College incorporated 
these feelings into a professional growth program, which is called 
Lifelong Investment For Excellence (LIFE). Also incorporated into this 
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program were instructor motivators and change process stages. These 
were incorporated in response to a revelation by Guskey (1985:5) that 
the "majority of programs failed because they do not take into account 
two critical factors: what motivates instructors and the process by 
which change . . . typically takes place." He (1986:7) also proposed a 
model for an instructor-acceptable professional development program: 
Staff Growth ---> Change in Teacher's Classroom Practice ---> Change in 
Student Learning Outcomes ---> Change in Teacher Beliefs and Attitudes. 
Changes in student learning outcomes and instructor beliefs and 
attitudes provide evidence that the growth program is effective and 
based on the evidence instructors will want to ^participate. 

A seventh factor identified in successful programs was 
continuation of the process. Effective growth programs must also 
involve a continual, spiral design to enhance this instructor self- 
actualization process. The LIFE program incorporates this idea (see 
Appendix F). The reason for this type of program approach is that 
learning new and/or modifying existing instructional techniques is 
complex. Wildman and Niles (1987:5-6) interjected that "the learning 
of a complex topic involves the cyclical interplay of three independent 
learning mechanisms: accretion, restructuring, and tuning." They 
defined the three mechanisms: (1) accretion--as the "straightforward 
accumulation of knowledge"; (2) restructuring—as the "creating of new 
memory structures"; and (3) tuning--as the "increments in speed, 
elaboration, flexibility, smoothness, and the like." They noted a 
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great deal of time is required in this process, especially in the 
"tuning" mechanism. 

The spiral denotes a continuous-directed program in h content 
of ever-changing cultural, economical, political, anu societ:'. condi- 
tions. While working with the Education Coiranission of the States, Ross 
and Solomon (1985:9) pointed out its importance as the "context will 
provide new demands and new expectations on teachers if goals, 
objectives, standards, criteria . . . remain static, the result can 
only be another crisis. ' Cross (1977:12) initially alerted educators 
to the concern when she stated, "It is understandably threatening to 
have your old role yanked out from under you before you have a new role 
to replace it." The spiral approach has a growth rather than remedia- 
tion emphasis. Instructors have been found to be more likely to 
participate in a growth program than a remedial one. 

Content of Professional Growth Pro grams 

•"'imber of content areas have been identified as valuable for 
inclusion in a professional growth program. While there is not com- 
plete agreement of wording of items, groupings, etc., by au+hors in the 
literature, there is consensus of the type of content to be included. 
In Guskey's Staff Development in the Process of Change (1986:/), 
primary consideration for inclusion of a goal or activity is reflected 
by his incorporation of a quote from Bolster (1983:298) in<;i sting that 
"ideas and principles about teaching are believed as true by teachers 
only when they tjve r^se to actions that work." The goals and 



42 



activities "must therefore, potentially result in improved student 
learning." Lanier and Glassberg (1981:26) suggested inclusion of 
teaching skills as they reiterated that "any and all such skills 
identified would become a basis for improvement in teacher education." 

Content areas suggested as making a difference in instruction 
have been the subject of a number of studies. A sampling of studies by 
Guskey and others in which a number of professional growth needs were 
identified as being useful in development of goals and activities 
(Guskey and Easton, 1983:266-271; McLean, 1986:5-6; Pedras, 1985:74-75; 
and Valverde, 1982:86-87) resulted in the following: 

1. Increasing student motivation. 

2. Reinforcing student learning. 

3. Accommoc ting different learning rates. 

4. Cooperation among colleagues. 

5. Characteristics of effective instructors. 

6. Course and curriculum development. 

7. Grading systems compatible with instructional objectives. 

8. Self-analysis of teaching skills. 

9. Developing course outlines. 

10. Writing test items. 

11. Writing instructional objectives. 

12. Diagnosis of learning and teaching problems. 

13. Application of lear ng principles. 

14. Course entry-exit level skills assessment. 

15. Selecting, developino. and using media 
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15, Advising and counseling students, 

17, Helping students to explore their motives, attitudes, and 

bel iefs, 

18, Techniques for evaluating instructional strategies, 

19, Use of computers, 

20, Utilizing group process skills, 

21, Use of community resources, 

22, Writing lesson plans, 

23, Identification of developmental education students, 

24, Textbook review and selection, 

25, Applying research findings to teaching, 

Identi fyi ng effective instructor characteri sti cs (el ements) 
has been thi objective of numerous studies. Results are applicable to 
many community college instructors. Studies have used alumni ratings, 
manager interviews, manager ratings, peer interviews, peer ratings, 
student achievement, student interviews, student ratings, and instruc- 
tor self-ratings as bases for determining effective characteristics. 
Some studies involved correlations, some involved factor analyses, 
while still others involved identification of common characteristics as 
the basis for conclusions reached. 

Education has been defined by r^emin (1977:viii) as a "delib- 
erate, systematic, and sustained effort to transmit, evoke, or acquire 
knowledoe, values, skills, or -sensibilities, as well as any outcomes of 
that effort," This definition is applicable to community college 
situations as the instructor's primary role in these institutions is to 
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teach, which Includes assisting students In setting yoa1s; helping them 
assimilate attitudes, knowledge, and skills necessary to reach their 
goals; and motivate them to achieve. Based on Cremln's definition of 
education, teaching "Is what we do to help students learn" (Fincher, 
1983:2-5) and Is a "dynamic, continuous Interaction between the teacher 
and student" (Haslett, 1977:44). Effective teaching can then be judged 
by amount of student progress on goals (Aubrocht, 1981:1). Heath 
(1982:35) defines an effective Instructor as one "Indeed empowering his 
students to become self-educating, compassionate, liberally educated 
people." 

importance and Use of Evaluation 
In Professional Growth 

Evaluations of Instruction are Important In professional 
growth programs to establish a base from which goals may be Identified. 
Evaluation provides a base from which Improvement proceeds. It also 
provides feedback Indicating th3 extent to which the Improvement plan 
Is being Implemented. 

"Evaluation of an Instructor Is not universally accepted. Is 
liked by no one, and Is even a threatening procedure regardless of how 
it Is approached," according to Miller (1974:7). Nonetheless, Miller 
quotid Priest (1967) as he supported evalUdtlon as "an Inherent element 
of any organized effort to achieve a goal." Green (1970) was used by 
Whitloy (1984:357) ^o promote evaluation, "Failure to evaluate Instruc- 
tion protects the Incompetent whTe falling to reward the competent." 
Nonldentlflcatlon of Incompetents enables them to remain Incompetent. 
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Through evaluation, needs of incompetents can be identified; and 
improvement begun. 

Some argued against evaluation of instructors for any reason, 
including growth on the basis of "assessment by others of what goes on 
in the teacher's classroom is an invasion of privacy" ''Mllman, 
1984:12). He argued, however, this shoulu not be the case because 
teaching is not a solitary activity affecting no one since "the lives 
of students are altered in far reaching and significant ways by the 
instructors with whom they react." Because of that impact, evaluations 
for u:e in professional growth are necessary. He also supported his 
contention on the basis of law as he stated, "Besides, court cases have 
made it clear that students have rights and that schools and colleges 
have a responsibility to ensure the quality of their curriculum and 
instruction." 

Some contended that the instructo"^ should not be evaluated 
because the evaluative process has not been shown to result in more 
student learning over time or that proper evaluation k just too 
complex and difficult (Gogan, 1985:11). Others also contended that 
evaluations should not take place because most types of evaluation are 
less than perfect (Stiggins and Bridgeford, 1984:11). They also stated 
that "while teacher evaluation practices are becoming more systematic 
procedurally, most are still insufficient to support viable teacher 
dev<»lopment programs." Andrews (1985:83) disagreed with this view as 
he emphatically stated, "This author contends that teaching can be 
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evaluated nd needs to be evaluated." Entrance into a faculty member's 
classroom for evaluation purposes is a necessary and positive move. 

Some argued against evaluation by oth rs saying that most 
would prefer to rely upon our own instincts and experiences for ongoing 
self-evaluation (Miller, 1974:9). "But," Miller cautioned, "such 
evaluation is limited by its nature." Instructors do not evaluate 
their own performance very effectively because of built-in biases. 

Some have also avoided evaluation because of its association 
with accountability and accompanying ramifications. This attitude, if 
present, needs to be overcome; and the purpose of evaluation explicitly 
shared. Andrews {1985:xi) was more explicit when he blamed murh of the 
fear of evaluation on a perception of "evaluation as something done by 
others to them." 

Faculty evaluation is important, and its results should pro- 
vide a basis for professional growth. Numerous authors have shown the 
importance of evaluation in professional growth. Licata (1985:1), for 
example, included evaluation in the "processes of livelihood and 
renewal." Others, including Andrews (1985:xi); Bolten (1973:99); 
Braskamp, Brandenburg, and Ory (1973:99); Fincher (1983:2); Koerin 
(1980:45); Ross and Solomon (1985:5); and von Glahn (1986:2), agreed 
that either the first or second most important reason for evaluation of 
instruction is to provide infor:nation for decision making in profes- 
sional growth programs. In fact, Andrews (1985:xi) indii:ated that 
during the last few years, evaluations of instructors has "become a 
common and expected occurrence." He concluded that instructors "may 
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perceive the action [evaluation] in a positive light ?s a way of 
enhancing their own performance." Barr and Krueger (1978:17) also 
promoted student evaluations as a "means for personal assessment and 
improvement of teaching." Bolten (1973:99) contended that "when a 
teacher views evaluation as a means to improve his instruction, he 
accepts it as a part of tho teaching assigiiment. " 

^^:.luation is necessary to establish a base from which 
improvement is to begin. This "starting point" (Bolten, 1973:99) ii 
necessary as a motivator because instructors will probably not learn 
what they think they already know or can do. Secondly, communication 
necessary for Improvement to proceed must be based on concrete data 
about the instructor's performance and "through this knowledge of 
strengths and weaknesses, a teacher can improve his work" (Bolten, 
1973:99). This occurs through the identification of goals and activi- 
ties for the individual's professional growth program as collected 
information provides answers to the question, "What kinds of faculty 
development would be most useful to me?" (Miller, 1974:11). Some 
improvement is possible if evaluation data is provided "to help the 
faculty examine their own teaching for improving it" (Braskamp, 
Brandenburg, and Ory, 1984:19). 

Evaluation must provide information of a "diagnostic nature" 
(Manns, 1985:271). This information then provides feedback necessary 
-0 enable improvement to continue as "human behavior is shaped, 
changed, and sometimes improved" (Davey and Sell, 1985:1). They also 
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indicate that feedback "can then be used to maintain, eliminate, or 
change the original performance." 

Importance of Student Ratings of Courses and 
Instructors in Professional Growth 

Although there are several sources of instructional evaluation 
data--peers, self, students, and supervisor—the "major, if not the 
only ... at most univvirsities" (Cranton and Smith, 1986:117) is that 
of students, tven information from students can be collected in a 
number of ways. These include interviews, rating scales, student 
achievement tests, and written appraisals (Braskamp, Brandenburg, and 
Ory, 1984:38). It seems that soliciting student ratings to assess 
effectiveness of the process is very appropriate and reasonable since 
the primary mission of any community college is the teaching-learning 
process (Rebalais and Durham, 1984:102). Goldschmid (1978:125) con- 
curred, "If teaching performance is to be evaluated ... a systematic 
m.asure of student attitudes, opinions, and observations can hardly be 
ignored. The data . . . :trongly suggest that use of formal student 
ratings provides a reasonable way of measuring student reaction." 

Many faculty members have questioned the use of student rat- 
ings because of concerns regarding competence of students to rate 
effectiveness of instructors. Age, background, experience, and extent 
of evaluation training of students are varied; so it would seem that 
evaluation results could also be varied and, therefore, not very -iseful 
in professional growth decisions. While this question has been in the 
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minds of many, a number of studies have been conducted to find answers 
to questions similar to this one. Goldschmid (1978:225) reviewed 
results and conclusions of a number of persons interested in finding 
answers to this question and concluded, "Most reviewers (e.g., 
Ale:nioni, 1974; Centra, 1973; Costin, et al., 1971; Doyle, 1975; Falk 
and Lee Dow, 1971; Floodage, 1974; Gage, 1974; Grush and Costin, 1975; 
Menges, 1974; Miller, 1972 and 1975; Murray, 1973; Seldin, 1976; Scott, 
1975; and Subkoviac and Levin, 1974) . . . have come to the conclusion 
that students are competent to rate instruction." Barr and Krueger 
(1978:18) also reviewed a number of studies and concurred with 
Goldschmid. 

More recent work has shown more use of student evaluations 
(ratings) to provide "speciric diagnostic daLa" (Eble, 1983:136). He 
also observed a substantial increase in the use of student ratings in 
just ten years, "Seldin's survey for 1981-82 shows that nearly 70 per- 
cent of the colleges reported using student evaluations as a major 
source of information about teaching ... up froui 35 percent ten years 
ago." Cohen (1980:321) referred to this increase as a "dramatic 
increase." Manns' (1985:271) studies indicated that student ratings of 
instruction were used by 88 percent of the two-year colleges. Cranton 
(1986:117) reported, "Student ratings of instruction are the major if 
not only component of the evaluation process at most universities in 
North America . . . results are used by faculty for the individual 
improvement of instruction." 
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Biehler (1978:41) argued in favor of student ratings because 
"in many respects, students are in i better position to evaluate 
teachers than anyone else." While it is true that students lack 
evaluation training, Biehler emphasized that "they know better than 
anyone else whether they are responding and learning. Furthermore, 
students form their impressions after interacting with a teacher for 
hours." The students do indeed serve as a major and direct source from 
whom information about instructional practices is collected (Davey and 
Sell, 1985:6). 

One explanation for the Increased use of student evaluations 
of course and instructor in professional growth is that studies have 
provided evidence that students are "discriminating judges" (Aleamoni, 
1984:112). He found that students frankly praised instructors and 
equally frankly criticized them. He stated that his results supported 
conclusions reached by others (Costin, et al., 1971; Frey, 1978; Gruss 
and Costin, 1975; Perry, ft al., 1979; and Ware and Williams, 1977). 

Ono distinct advantage of student evaluations is their 
practicality. They cost faculty members relatively little time, take 
in all students, in all classes, and include factors that relate to the 
students' perceptions of Icavning (Eble, 1983:137). He further states, 
"They [student evaluations] do provide one means by which faculty are 
better able to judge how they are engaging the attention of all 
students in a class, and engaging attention is basic to learning." 
Student rating instruments have been developed for ease in administra- 
tion and scoring. Centra (1972:31) used this factor to promote 
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continued use of student ratings "in view of the ease with which 
student ratings can be employed for instructor improvement." 

Student ratings have been used in a large number of 
institutions and are considered to l>e used in a systematic manner to 
improve instruction. In an extensive study (almost eight hundred 
instructors) conducted by Centra (1976:14), results indicated that over 
80 percent of the faculty perceived t.heir use to be at least moderately 
effective in professional growth. Elements rated by students are 
actually teaching behaviors. These behaviors are the ones addressed in 
professional growth programs because they do have an effect on student 
learning. Because of this, Whitley (1984:42) emphasized that "student 
rating scales are indispensable sources of information in evaluation of 
instructors" for professional development purposes. 

The value of feedback regarding need for change in the extent 
of progress being made toward desired changes has been accepted by many 
educators. Effectiveness of student ratings in providing an impetus 
for change has been sufficient to cause educational leaders to urge 
their continued use because "they appear to have sufficient impact to 
warrant continued use as one method of improving college teaching" 
(Centra, 1972:31). As he continued to study impacts of student 
ratings. Centra (1977:96) found that change was not necessarily a slow 
process once instructors became aware of differences between their 
perceptions and student perceptions. He found that many instructors 
were able to set goals and make adjustments in as little as half a 
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semester after receiving rating results. Over a longer period of time, 
even more instructor^ made some poritive changes, 

A distinct advantage of student ratings over peer or 
supervisor evaluations is the relative lack of previous data and per- 
sonal instructional biases, O'Hanl^n and Mortensen (1980:670) conceded 
that students have biases, but that these biases are usually not 
polariZ3d in a classroom. They contended, however, that students do 
not have a "bias due to previous data, personal relationships, reason 
for observation, own philosophy and \alues, or favored teaching 
methods" which the supervisor has that may interfere with analyzing and 
interpreting the data objectively. 

Though use of student evaluations--student ratings in particu- 
lar—of courses and instructors is increasing for reasons presented, 
there is opposition to their use because of inherent limitations. 
Instructors are able to manipulate the class setting; and, therefore, 
student rating results may not be indicative of the entire semester 
(Kallison, 1986:345). Because of po*:sible manipulation, he urged 
educational leaders to keep these limitations in mind as they assist 
instructors in professional growth programs to ensure a direction 
accurate and reflective of instructor needs. 

Instructors and supervisors or other staff growth persons 
working with instructors need to heed another limitation of student 
ratings. Direct comparisons are not possible whenever two instructors 
are teaching different courses, McKeachie (1983:37) reminded educa- 
tional leaders that "we are comparing apples and oranges even though we 
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have numerical ratings which appear to be directly comparable." He 
used an example to illustrate the point. "Simply obtaining a mean 
rating of 2.1 for one teacher and a 2.2 for the other does not mean 
that their teaching has magically become directly comparable." 

Centra (1972:30) emphasized the need for "comparative or 
normative data to help the teacher understand better his or her 
students' rating." Currently emphasis is placed on comparative data 
over time for providing feedback on instructor progress toward meeting 
professional grov/th goals. The change in emphasis from comparative or 
normative data from student ratings to diagnostic data has occurred 
during the last fifteen years. Emphasis has also been placed on diag- 
nostic data for determining individual goals for the instructor. 
Johnson (1984:90) indicates his disagreement with Centra as he stated, 
"Centra's ninth guideline (use comparative data) is only acceptable to 
me when used with multiple sections of the same course and when we can 
be sure that the students are also comparable." Diversity of students 
and courses is responsible for the change of emphasis. These condi- 
tions severely limit the value of comparative data. 

Because of absences, completing courses early, etc., it is 
highly unlikely that all students enrolled in course will actually 
complete a student rating instrument for that course or instructor. 
This could lead to a limitation known as "response bias resulting from 
divergent raters" (Centra, 1980:28). He suggested reducing this 
limitation by having a sufficient proportion respond, stating 
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"two-thirds of the enrolled students in a course should be the minimum 
desirable proportion." 

One limitation of earlier student rating instruments was the 
nonspecific nature of the items. This limitation was addressed by a 
number of persons in the 1970s. Educational Testing Se'^vice's (SIR) 
and Kansas State University's (IDEA) forms are two examples of a diag- 
nostic-type instrument. Hunter (1983:3) concluded, "A number of teach- 
ing evaluation instruments have been developed that are more fully 
systems than the older variety of forms--that is, they enable a match- 
ing of cognizant styles and teaching methods." The current instruments 
provide more detailed and specific information for use in professional 
growth programs. 

Reliability and Validity of Student Ratings 
of Courses and Instr uc tors 

A number of characteristics regarding student rating data were 
identified as important in determining usability and effectiveness of 
that data. These included "dimensionality, reliability, validity, 
usefulness, and susceptibility to bias" (Marsh, 1984:342). Both 
instrument and process must have credibility to both instructors and 
administration to gain acceptance and be effective in professional 
improveme t (Braskamp, Brandenberg, and Ory, 1984:23). Because of 
interest in acceptance of the student rating process and its results, 
"thousands of papers have been published about them" (Braskamp, 
Brandenberg, and Ory, 1984:243). After an extensive literature review, 
they noted conclusions similar to those already reached, "Student 
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rating instruments and data are reliable and valid." They also founJ 
inconsistencies as 'suspected sources of bias" that needed to be 
addressed. 

Correlations of four dimensions between midsemester and end of 
semester ratings ranged from +.70 to +.87 in a study conducted by 
Costin in 1968 (Costin, Greenough, and Menges, 1971:512-513). These 
dimensions were skill, structure, feedback, and rapport. The dimension 
of group interaction had a correlation of +.48. They reported that 
Costin (1971) obtained correlations of +.67 to +.77 for dimensiops of 
student involvement, instructor support, instructor control, and 
negative affect in a study he conducted. 

Time periods involved in the correlation studies varied con- 
siderably, which provided additional support for reliability and 
validity levels. Centra (1980:28) reported that tcstin (1968) and 
Centra (1972) obtained correlation results over relatively short time 
periods--half semester and fivi weeks, and Overall and Marsh (1978) 
obtained significant correlations for student ratings coll'jcted one 
year apart. Aleamon- (1984:112-113) reported that Marsh and Overall 
(1979) and McKeachi- et al . , (1978) obtained results th.-t were very 
simila>' to further substantiate earlier ^'indings. These results were 
obtained from alumni who had been out five to ten years. Feldman 
(1978:200-201) reported similar correlations in his list of citations 
and results of his study. Limits of estimates of reliability can more 
"ofter, be in the .80s and .90b--when the ratings of at least twenty to 
twenty-five students in the same classroom are averaged together." 
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Centra (1980:25) issued a similar directive regarding numbers of 
students. Blackburn and Lawrence (1985:272) also repo'-ted test-retest 
rrtliab^ities of over +.90 in a recent study. Tomasco (iii80:79) 
reported on a review of literature and concluded that more recent 
literature "would seem to confirm the earlier conclusions of Costin, 
Greenough, and Menges (1971) and others regarding rating consistency." 
Specifically, results reported by Carrier, Howard, and Miller (1974); 
French-Lazovik (1974); and Murray (1975) were cited. 

Internal consistency studies have been reported it least as 
far back as 1954 according to Costin, Greenough, and Menges (1971:513). 
Cor-2lations in the range of from +.77 to +.94 were reported by: 

1. Guthrie (1954)--involved random student pairs. 

2. Maslow and Zimmerman (1954)--invol ved random student 

pairs. 

3. Spencer (no date)- -involved random student pairs. 

4. Lowell and Homer (1955) --involved odd item means versus 
even item means. 

5. Spencer (1958)--invol ved negatively stated item means 
versus positively stated item means. 

Even higher internal consistency reliabilities of +.81 to +.98 were 
reported by Aleamoni (1984:113). 

. After reviewing student rating studies conducted over the last 
twenty years, Murray (1983:138) concluded that "the weight of evidence 
suggests that student ratings of a given instructor are reasonably 
stable across course and time periods." He also concluded that ratings 
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"are affected to only a minimum extent by extraneous factors [elements] 

such as class size and severity of grading"— the two factors most often 

listed as concerns by instructors. 

The review of the literature at this point in time regarding 

well -constructed student rating instruments seems to be no different 

than in 1971. Therefore, the same conclusion regarding reliability 

made by Costin, Greenough, and Menges (1971:513) is appropriate today. 

They concluded that 

. students can rate classroom interaction with a 
reasonabl e degree of rel i abi 1 i ty , in particul ar the evi dence 
cited concerning the stability of student ratings argues against 
the contention (sometimes made by opponents of student ratings) 
that student o^ inions of instruction are difficult to interpret 
since they might be made after a particularly good or bad 
atypical experi ce (e.g., a lecture). 

The validity of an instrument is important in its acceptance 
for use in professional growth as well. It is important to know that 
student ratings correlate with other measures of effective teaching 
factors and ti.dt students ran really judge when they are learning. 
Studies reported in the literature contain a considerable amount of 
evidence regarding the high level of validity for student ratings. 
McKeachie (1983:37-38) reviewed Cohen's (1981) study on validity 
involving sixty-eight courses which reported a validity coefficient of 
+.40. Ho assured us, "This is much higher than we would expect and is 
probably very reassuring to those who are concerned abn^^t whether stu- 
dents can judge when they are learning." 

Miller (1974:9) reported on a study conducted at the Center 
for Research and Development in Higher Education by Hildebrand and 
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Wilson (1970:3) that disclosed, "There is excellent agreement among 
students and between faculty and students about effectiveness of given 
teachers." White (1976:122) reported results of a study that had a 
"positive though moderate correlation o? +.5386; significanc at the 
.023 level" between student evali* cions of instructors and learning. 
Concerned that questions might arise because of differences of ability 
of students in the fourteen-course sections, corrections were then made 
for cumulative grade point averages of the students. This procedure 
resulted in a corrected positive correlation of "+.4941; significant at 
the .043 level." 

Dowell and Neal (1982:61) reviewed a number of validity of 
student ratings of instructors versus student achievement and concluded 
that "literature indicates statistically significant but very modest 
validitv coefficients." Cohen M981: 281) performed a meta-analysis on 
forty-one independent validity studies that related student ratings of 
instructors with student achievement. The average correlation between 
student overall ratings of the instructor and stu t achievement was 
+.43 and between student overall course rating and student achievement 
was +.47. He concludtd that results of meta-analysis "provide strong 
support for the validity of student ratings as measures of teaching 
effectiveness." 

The third determinant of validity level was correlation 
between supervisor ratings and student ratings. While this 
correspondence was established over twenty years ago, it seems to be 
currently accepted. Costin, Greenough, and Menges (1971:516) referred 
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tc a study done by Costin (1956) which resulted in "a significant 
correlation of +,49 between the mean score of all items of student 
ratings and chairman's ratings of overall effectiveness 

Eble and Berg (1976:13) critically reviewed several reports. 
One was by Rodin and Rodin ;i972) in which they reported "that the 
students' evaluations tend merely to reflect the personal and social 
qualities of an instructor," Based on research findings, Rodin and 
Rodin concluded that "students rate most highly instructors from which 
they learn least." A similar conclusion was reached by Naftulin, Ware, 
end Donnally (1973:534) as a result of a study that has come to be 
known as the Doctor Fox Effect, This conclusion was based on a study 
in which an actor was hired to teach one section, and regular 
instructors taught other sections of the same course for a very short 
period of time. This has become known as the Doctor Fox Effect because 
of the name given to the actor. Rejection of this almost exactly 
opposite result from those of other studies was not based on 
methodology or statistics but on the basis that students cannot be 
fooled over a long time about an instructor's competence or level of 
student learning. Another study reviewed by Eble and Berg was one by 
McDaniel (1972) in which there was no confirmation of the lack of 
validity concluded by Rodin and Rodin. McDaniel correlated instructor- 
student agreement as a basis for that conclusion. 

Another method for determining validity of student ratings was 
determination of the correspondence between students and instructors as 
to what constitutes effective teaching. It was assumed that if the 
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resuUs were to be valuable in professional development, data should be 
based on the same operational definition of effective teaching. 
Goldschmid (1978:224) reviewed validity studies involving this rela- 
tionship. These studies conducted by Aleamoni and Ymer (1973) ; 
Greenwood, et al • (1973); and Hildebrand and Wilson (1970) found "a 
great deal of agreement between students and faculty on what consti- 
tutes effective teaching." 

Construction of the instrument is important if credibility for 
use of results is to be maintained- Correlations of over +.90 for 
well -constructed instruments have resulted. High correlations indicate 
that students do ta! e completing the instrument seriously and that 
students can differentiate entertaim^ent from a sound performance 
(Blackburn and Lawrence, 1985:271-272). 

Importance of Course, Instructor, and Student Elements; Stude nt 
Demographics; and Course and Program Classifications in a 
Student Rating of Course and Instructor Instrument 

Authorities agree that course, instructor, and student 
elements need to be included in any student rating instrument that is 
to provide the basis for an individual instructor's professional 
development program. Furthermore, wording lust be specific if desired 
goals for improvement are to be identified and activities selected for 
improvement. Goldschmid (1978:229) reviewed the reports of Frey 
(1976), Menges (1974), Pohlman (1975), and Sherman and Winstead (197^^ 
and concluded that feedback must "indicate precisely what instructional 
elements require modification," 
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Marsh {l<j78:5-8) provided a list of elements to be included in 
an instrument ured in student ratings of course and instructor. The 
following were listed: (1) learning value, (2) instrument enthusiasm, 
(3) organization, (4) group interaction, (5) individual rapport, 
(6) breadth of coverage, (7) examinations, (8) assijnments, (9) le.>l 
of interest, (10) work load/difficulty, (11) overall GPA, (12) percent 
students enrolled in same division as course, (13) expected grade, 
(14) reason for course and (15) class level. 

Centra (1980:19) classified elements as "(1) organization, 
structure or clarity; (2) teacher-student interaction or rapport; and 
(3) teaching skill, communication, or lecturing ability." Carter 
(1982:7) added elements "normally considered to include statements of 
objectives, advance organizers, content outlines, segments of content, 
study questions and exercises, se":f-tests, etc." In addition, a need 
was expressed for ample procedural directions and explanations to 
ensure easy student movement from activity to activity. 

Washton (1983:5) reviewed three studies to determine their 
support for student interactions in a classroom setting for increased 
learning. These studies, conducted by Lysakowski ?.nd Walberg (1982), 
Noll and Allen 2), and Ostertnan (1982), p-ovided data that lead to 
the conclusion .hat "student participation before, during, and after 
lectures" is important. 

Cohen (1980-331) reviewed several studies to determine ele- 
ments to be included in a student rating instrument as well. Various 
studies indicated a variety of elements: 
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1. Kulick and McKeac'-.ie {1975)--elements of skill, rapport, 
structure, and difficulty. 

2. Isaacson, et al . (1964)--elements of interaction and feed- 
back. 

3. Cohen (1977) --elements of student learning progress, 
student attitude toward subject matter, and student achisvement. 

The course, instructor, and student elements need to be specific and 
small enough in scope so that each can be dealt with in the improvement 
process. Braskamp, Brandenberg, and Ory (1984:53) emphasized need for 
specifics in "specific and diagnostic items are the most appropriate 
items because they attempt to measure specific instructor behaviors or 
course characteristics." Omaggio (1982:261) also developed a student 
rating instrument. A number of elements were iriluded as items in that 
instrument. The following elements were included: (1) organization of 
class meeting, (2) instructor's interest in subject, (3) knowledge of 
subject matter, (4) quality of skill in using subject, (5) clarity of 
explanation, (6) interest in class session, (7) freshness of presenta- 
tion of instructor, (8) tolerance and helpfulness, (9) classroom 
activities, (10) promptneos of returning homework, quizzes, etc., and 
(11) feedback on homework and quizzes. 

Bantz and Rodgers (1985:270) also listed a number of factors 
(elements) that should be considered for inclusion on a student rating 
Instrument. Their 'tsting included: (1) course difficulty, (2) course 
structure, (3) intensity of teaching, (4) course work load, 
(5) personal life-style, (6) classroom atmosphere, (7) fairness in 
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grading, (8) enthusiasm, (9) interest as lecturer, (10) dynamic/ 
charisma, (11) corraiiunication skills, (12) personal appearance, 
(13) influence on students, and (14) relation with students. 

Coi relations and Influence of Course, Instructor, and 
Student Elements; Student Demographics; and Course 
and Program Classifications on Studint Ratings 

Opponents of stuaent ratings have conducted numerous studies 
in efforts to obtain data that would support their beliefs that student 
ratings were biased or could be biased as a result of "invalidating 
influences" (Cashin, 1983:595) of the course, instructor, or student. 
Others have conducted studies to provide evidence that the students' 
ratings could not be unduly influenced by these elements. In all, 
"potentially biasing factors [elements] have been the subjects of 
hundreds" of studies and articles (Cohen, 1981:281). Twenty-four such 
elements were selected and are included with citations in the discus- 
sion to follow. The following elements are included: (1) attitudes of 
"i.dssmates, (2) class size, (3) course difficulty, (4) course function/ 
type, (5) course level, (6) course objectives clarity, (7) course 
objectives and content agreement, (8) feedback to students, (9) grading 
practices, (10) instructor age, (11) instructor experience/rank, 
(12) instructor interest in students, (13) instructor knowledge of sub- 
ject, (14) instructor openness to viewpoints of others, (15) instructor 
personality, (16) instructor presentation skills, (17) instructor 
publications, (18) instructor respect for students, (19) instructor 
gender, (20) instructor-student rapport, (21) organization of materials 
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and presentation, (22) student achievament/grade/performance, 
(23) student gender, and (24) subject area. 

1. Attitudes of classmates--Miller (1974:93) reported a 
correlation of +.72 with the overall rating of an instructor. This was 
significant at the .01 level. No influence on the overall rating of an 
instructor score by this element was suggested, however. 

2. Class size--Feldman (1978:206) reviewed ninety-five 
studies relating class size to overall rating of an instructor. He 
reported that about one-third (nearly thirty studies) found essentially 
no relationship between class size and overall ratings of an instructor 
and in roughly two-thirds, found indications of a negative relation- 
ship. Correlations ranged from -.10 to just under -.30. Marsh 
(1982:487) reported that Bausell and Bausell (1979) found that students 
in a course with larger enrollments tended to rate an instructor "less 
favorably." He also reported on a study conducted by Marsh, Oberall, 
and Kesler (1979) in which they concluded that the negative relation- 
ship between class size and overall rating of an instructor "was 
limited primarily to ratings of [elements] Group Interaction and 
Individual Rapport" and not the overall rating of an instructor. They 
reported that Frey (1978) also obtained similar results. O'Hanlon and 
Mortensen (1980:669) concluded that class size influenced the overall 
rating of an instructor, but the correlation was not significant. 
Whitley (1984:42) found that there was some rpgative correlation 
between class size and overall rating of an instructor. Because the 
differences in class sizes; e.g., eight to ten versus thirty to fifty 
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or more before there was significance, were so great; and because this 
was considered an element "beyond the control of the teacher," ho 
questioned the value of further study of the Influence of class size on 
rating of an Instructor. 

3. Course d1ff1culty--Course difficulty has been studied as 
an element suspected of having a negative effect on student overall 
rating of an Instructor. Evidence collected Indicates the difficulty 
of the course Is not Important In lowering an overall rating of an 
instructor. Although Bantz (1975:267) reported results of some lower- 
ing of the students' overall rating of an Instructor with the more 
difficult courses, Braskamp, Brandenburg, and Ory (1984:47); Centra 
(1977:20); and Haslett (1977:50) reported that course difficulty had 
one of the smallest negative correlations with the overall rating of an 
Instructor. 

4. Course funct1on/type--Aleamon1 (1984:113-115) reported 
that results Indicated that student overall ratings of Instructors In 
required courses were lower than In elective courses. Blackburn and 
Lawrence (1978:272) reported "slight biases of higher ratings 
[Instructor ratings] going to elective (over required) courses." 
Andrews (1985:83-84) reviewed results of studies conducted by Centra 
and reported that "students tend to rate Instructors of elective 
courses or courses In the major more highly than courses taken to ful- 
fill a college requirement." Lein and Mertz (1976:3) reported on a 
study conducted by Behling and Mertz In which they concluded that "an 
instructor's evaluation does not depend on whether he or she teaches an 
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elective or required course." Aleamoni {lf>84:113-115) ; O'Hanlon and 
Mortensen (1980:669); and Stumpf, Freedman, and Aguanno (1979:120) 
reviewed literature and concluded that students rate instructors in 
elective courses "to a minor extent higher" than in required courses. 

5. Course level --A very limited number of studies have been 
conducted to detemiine the correlation between the level of the course 
and the student's overall rating of an instructor. These have obtained 
data indicating various correlations. Aleamoni (1984:113-115) reported 
that no conclusion could be made because of the variety of the results 
obtained. Aleamoni (1984:114) reviewed research studies relating to 
the question and cited eight researchers who "reported no significant 
relationship between student status (e.g., freshman, sophomore, etc.) 
and ratings assigned to instructors." Lein and Merz (1976:3) concluded 
after reviewing several studies that "an instructor's evaluation does 
not depend on whether he or she teaches introductory or advanced 
courses." Marsh (1980:236) also concluded that the level of instruc- 
tion did "not seem to make much difft ence" in the student's evalua- 
tions of teaching effectiveness. 

6. Course objectives clarity--The course objectives must be 
clarified in order for the student to understand the intent of the 
instruction and the achievement expected. Review of research results 
supports a positive correlation between the clarity of course objec- 
tives and the overall rating of an instructor. Centra (1972:5) 
reported significant differences between student overall ratings of 
instructors who presented clear course objectives to their students and 
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those who did not. Miller {1974:Appendix B) reported a correlation of 
+.68 between clear course objectives and student's overall rating of an 
instructor which was significant at the .01 level. 

7. Course objectives and content agreement --Several 
researchers have conducted studies that support course objectives and 
content agreement as a positive influence on student ratings of 
instructors. Centra (1972:5) found there were "signficant differences 
. . . [in the] amount of agreement between objectives and what was 
taught" and the overall rating of an instructor. Miller {1974:Appendix 
D) also supported this expected correlation eporting a correlation of 
+.67 which was significant at the .01 level. 

8. Feedback to students- Very few studies seem to have 
addressed this element. The results, however, suggested it is not 
whether or not feedback is provided but rather the type and manner in 
which the feedback is given that is important. Tomasco (1980:81) 
summarized results of the review of studies in this way: "Teachers 
with favorable evaluations . . . provide feedback in a humble, non- 
authoritarian style." Even here the significance of the difference 
between the groups of instructors is questionable because the results 
seem to be inconclusive. 

9. Grading practices--Several studies have dealt with the 
grading practices of the instructor. Centra (1972:5) found a signifi- 
cant difference between the rating of instructors who informed students 
how they were going to be evaluated and graded and those that did not, 
but not between those instructors in each group using different methods 
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to evaluate or grade students, Ha5^1ett (1977:50) noted some 
consistency in the correlations found between an instructor's evalua- 
tion method ar *:he student's overall rating of an instructor, but the 
correlations were not significiint, Lein and Merz (1976:3) reported 
that the results of Behling and Mertz' study indicated that "imposlnq 
academic rigor upon students will not in itself result in poorer stu- 
dent evaluations of the irstructor," Lichty, Vose, and Peterson 
(1978:10) disputed the argument of some that state that instructors can 
inflate grades to maintain or improve the overall instructor ratings. 
They argued that "if stude' .s perceive a university education as an 
inferior 'Giffen/ good student*; will reduce their overall [instructor] 
evaluations," Review of research studies of the late 1970s by 
Tolefson, et al • (1980:1-2) indicated that McKeachie (1979) and Palmer, 
Carlines, and Romer (1978) found no significant correlation between 
grading practice and the overall rating of an instructor, 

10, InstrucLor age--The element of instructor age was also 
listed as a concern because there seems to be a general question about 
how well older instructors are able to relate to students. This may 
have been more of a concern previously than it is now because the age 
range of ^'udents in courses at the two-year postsecondary institutions 
is increasing, A number of studies have been reported on this element 
with mixed results, Genova, et al,, (1976:29) reported that Bryson 
(1974) "found a positive but not significant correlation" while Koerin 
(1980:44) concluded that age apparently did "not correlate with judg- 
ments of teaching effectiveness," 
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11. Instructor experience/rank- -Centra (1972:18) reported 
results indicating that evic nee did r.ot support the assumption that 
student overall ratings of instructors correlate with ranks of 
instructors. He stated that "differences were not significant." 
Genova, et al., (1976:29-30) concurred with that assessment as they 
stated, "The age, experience, and rank of the teacher had been found to 
be positively correlated to student ratings"; however, the differences 
were not found to be significant. Stumpf, Freedman, and Aguanno 
(1979:120) concluded that "external variables [instructor experience 
included] may have some minor impact on the ratings of instructors, but 
they do not explain much of the variance in student ratings or faculty 
performance." Reported results in the litera* i did not indicate 
significant correlations between instructor experience a.;:. sLudent 
overall ratings of instructors. 

12. Instnctor interest in students--Several studies were 
conducted to determine the extent of f-is expected relationship. 
Centra (1972:5) and Tomasco (1980:81) reported c significant difference 
for this element. Hunter (1982:7) suggested th<-.t "as evaiuators, uhey 
[students] usually are generous toward any teacher who is genuinely 
interested in their welfare and trying his best to create a learning 
climate." This would seem to explain why there is a positive relation- 
ship between the instructor'*; interes* in students and the student's 
overall rating of instructor. 

13. Instructor knowledge of subject--The genera' expectation 
is that this element would have a positive influence on the student's 
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overall rating of the instructor. Only one study was identified that 
dealt with the influence of *his element. Haslett (1977:50) concluded 
that instructor competence is shown to consistently influence student 
overall ratings of instructors. However, student level of competence 
seems to have no correlation with the overall rating of the instructor. 
This Ko-ld indicate that instructor knowledge is an important element 
whether or not that knowledge has been transferred to the student. 

14. Instructor openness to viewpoints of others--Openness to 
viewpoints other than one's own \ ..uld seem to be positively correlated 
with overall ratings of an instructor. Only one study was found that 
dealt with openness of an instructor to viewpoints different from the 
instructor's. Centra (1972:5) reported that there was a significant 
difference in the correlations between the various levels of openness 
to ideas of others and the students' ov. 'all ratings of instructors. 

15. Instructor personal ity--Instructors with certain person- 
ality types could exhibit behaviors that have an effect on student 
feelings, and students could respond to this effect through either a 
lower or a higher rating. A study conducted by Murray (1975:58) 
supported this contention. Four personality types identified in his 
study, leadership, extroversion, objectivity, and lack of anxiety, 
accounted for two-thirds of the between- instructors variance in student 
ratings of instructors. Toriasco (1980:81) presented results that 
"confirm these assumptions and suggest the teachers with fivorable 
evaluations have specific personality characteristics which students 
consistently identify." 
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16. Instructor presentation skills--It is reasonable to 
expect a positive correlation between presentation skills exhibited and 
the overall ratings of the instructor exists. Braskdiiip, Brandenburg, 
and Orv (1984:47) reported a moderately positive correlation between 
the level of teaching skills exhibited and the overall ratingr. of 
instructors. Genova, et al., (1976:30) and Omaggio (1982:266) reported 
correlation results from their studies ranging from +.57 to v. 91, and 
Omaggio concluded that "the more effective teacher ... is one who 
tries to incorporate such personalized language into the daily lesson 
plans." Concepts and ideas must be presented so the students can 
understand them. Because students havt been shown to be discerning 
individuals, it should follow that clear explanations by the instructor 
would influence the overall ratinq in a positive manner. A study of 
the degree of explanation of ideas was also found to have a significant 
positive cor'^elation of +.68 with the overall ratings of Instructors 
(Miller, 1974:93). 

17. Instructor publications--An increased number of publica- 
tions would tend to provide additional information and experience for 
use in the classroom. There is a question as to whetiif^r higher overall 
ratings would also occur. The results of studies do not support higher 
ratings to any extent, "he number of publications "may have some minor 
impact on the rating of instructors, but they do not explain much of 
the valance in student ratings of faculty performance" (Stumpf, 
Freedman, and Aguanno, 197P:120). 
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18. Instructor respect for students-- Centra (1972:5) 
supported an assumption that respect for students and overall rating of 
the instructor would have a high positive correlation. He found a 
significant difference when the instructor respected student's ability 
enough to expect them to "think for themselves." Tomasco (1980:81) 
confirmed this assumption as well when he obtained results that indi 
cated a high positive correlation when instructors had "r2spect for 
student opinions," 

19. Instructor gc^der--Results of a study by Costin 
(1971:520) indicated that gender of an instructor had no influence on 
the stude-it overall ratings of instructors. Subsequent studies by 
Centra and others (Centra, 1972:18; Elmo^-e and LaPointe, 1975:370; 
Haslett, 1976:53; and Stumpf, Freedman, and Aguanno, 1979:12) resulted 
in similar findings. No significant difference was found between 
overall ratings given to female instructors and those given to male 
instructors regardless of the gender ratio of students in th: course. 

20. Instructor-student rapport--Several studies were con- 
ducted to determine if a relationship between instructor-student 
rapport and student overall ratings of instructors could be found. 
Bantz (1985:2C7) reviewed the literature reported by Centra (1979), 
Cohen (1981), Dowell and Neal (1982); and Marsh (1980) and used their 
results as a basis for concluding th .t "the results of these studie'; 
were highly suggestive." However, results and analyses of her own were 
quite definite with "the resuHant squared multiple coefficient" of 
.09. This negligible relationship definitely did not support any type 
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of relationship between instructor-student rapport and student ratings. 
Haslett's (1977,51) review of studies conducted by Cashin (1974) sup- 
ported the consistency of the evidence as results indicated some 
positive correlations with overall ratings of instructors which were 
not, however, significant. She concluded, "This adds more support to 
the claim that student evaluations indeed measure and reflect teaching 
effectiveness and not popularity." Popularity is an indicator of 
student-instructor rapport, not teaching effectiveness. 

21 , Organization of materials and presentation- -Several 
researchers have conducted studies to determine if the evidence found 
to determine the correlation between how well the materials and 
presentations are organized and student overall ratings of instructors. 
Bantz (1985:267) reported that one of the factors affecting students' 
rating of instructors was "organization." Braskamp, Brandenburg, and 
Ory (1984:43) reported a correlation of +.80, which they termed 'fairly 
hirh," and, therefore, a factor to consider in student overall ratings 
of instructors and in diagnosing a low overall instructor rating. 
Miller (1974:Appendix B) reported a corr " jn of +.67, which was 
highly significant (at the .01 level), between organization and 
planning of class presentations and student overall instructor ratings. 

22 . Student achi evement/grade/perf ormance- -A number of 
studies found positive j elatlonships between the grade given to 
students and their ratings of instructors. Barr and Krueger (1978:18) 
indicated a contradiction when he reported a number of highly signifi- 
cant correlations between expected grade and Student Evaluations of 
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Teaching (SET) ratings but admitted chat the "literature presents mixed 
conclusions" even though not significant "in almost all cases the 
correlations between grade and teacher rating yields a positive 
coefficient," In most studies he reviewed, the correlation coefficient 
is approximately +,20, which was interpreted as significant or not 
significant depe^^ding on tf a number of subjects in the study, Ba'isell 
and Magoon (1972:102) admitted the results of their study differed from 
most others in that a "significant relationship was found to e;^ist 
between student grade and rating gi/en the instructor," Braskamp, 
Brandenburg, and Ory (1984:47) reported a moderate'iy positive correla- 
ti on between grade and rat i ngs of i nstructors , Snyder and CI ai r 
(1976:81) reported thac students ho obtained higher grades "rated the 
instructor as a better teacher overall," Penfield (1978:20) reviewed a 
number of stuoies conducted over a long period of time and concluded 
that "there appears to be a slight positive relationship between 
expected grade in the course and student overall teacher ratings. This 
trend seemed to ha \ occurred in the last ten years," Hoffman 
(1978:291) reported generally low but statistically significant 
correlations," Costin (1971:518-519) reviewed a number of studies and 
found significant relat-^onships even though the coefficient was usually 
weak, less than ,30 in Anikeef (1953), faffrey (1959), Echandia (1954), 
Elliot (1950), Rayder (1958), Rubenstein and Mitchell (1970), Russell 
and Bendig (1953), Spencer (1958), Stewart and Malpaso (1963), 
Treffinger and Feldhusen (1970), Walkc;' (1959), and Weaver (1950), 
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Other studies Indicated a tendency for almost ..o Influence by 
the grade received by the student* Aleamoni (1984:115) reported a none 
to weak positive correlation between grades received and overall 
in.tructor ratings. Garverick and Carter (1962:213) reviewed the 
results of a number of studies and concluded, "Research on this subject 
has not settled the Issue. . . . some bias In the direction of 
expected grade was found." Frey (1973r85) reported In a rebuttal that 
•'there was no evidence of a strong positive relationship between final 
exam grades and the [students'] ratings." , 

Costin, Greenough, and Menges (1971:518-519) conducted a 
review of studies and also found no relationship between student grades 
and student ratings of instructors. They observed no relationship In 
the following studies: Bendig (1953), Blum (1936), Cohen and Humphreys 
(I960;, Eckert (1950), Guthle (1949 and 1954), Hellman and Armentraut 
(1936), Hudelson (1951), Remmers (1928, i:30, 1939, and 1960), Russell 
(1951), and Voeks and French (1953). In a more recent study, Blackburn 
and Lawrence (1986:272) concluded, "Scores are not related to the grade 
that the student expects to receive or does get." 

The previous studies related the actual grades receved to the 
student ratings of Instructors. One group of researchers was inter- 
ested not In the actual grade but In the effect of a student receiving 
a grade different than that expected. Tollefson, et al . (1980:1-2) 
reported mixed results after reviewing a number of studies, but a 
statistically significant correlation was fonrd when a grade lower than 
expected was given to the student. 
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The element (grades given to students) has probably been 
studied more than any other element to determine its effect on a 
student's overall rating of an instructor. The results of the studies 
have been varied and inconclusive; however, there seems to be support 
for a tendency toward a positive correlation between the student's 
achievement or actual or expected grade and the student's overall 
rating of the instructor. There is also support for a negativ? corre- 
lation between achievement or grade and the student overall instructor 
rating when the student's achievement or grade was less than expected. 

23. Student gender--The gender of the student making the 
o>/arall ratings of instructors seems to have little influence on the 
ratings according to the literature. Costin (1971:520) reviewed a 
number of pre-1970 studies and reported no significant influence of 
gender of the student or the gender of the instructor on overall 
instructor ratings. These included Bendix (1953), Caffrey (1969), 
Downie (1952), Elliot (1950), Heitman and Armentrout (1936), Lovell and 
Haner (1955), and Renmers (1936). In only one study (Benriix. 1952) was 
there a tendency toward increased overall rating of female instructors 
by female students, and this tendency was not significant. 

Elmore and LaPointe (1975:370) and Haslett (1976:53) reported 
no significant difference between overall ratings of instructors from 
students of the same or opposite sex. When Elmore and LaPointe 
examined individual items, they ^ound that in only one item, "showed an 
interest in students," was there a significant difference in the 
ratings "female students rated female instructors higher, and male 
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Student^ rated male instructors higher. Barr and Kruejer (1978:22) 
reported that no influence of student's gender was. found in the SET 
results. When student ratings were compared for the instructor of the 
same or opposite gender of the student, there were conflicting reports 
of correlations. According to Aleamoni (1984:115), there did not 
appear to be any significant difference in student ratings of 
instructors of the same or opposite gender of the student. 

24. Subject area- -There does not seem to be a correlation 
between subject area and student's o arall rating of the instructor. 
Only one .study was identified that addressed the question of 
cor-^elation between subject area and the overall rating of the 
instructor. In this study, the results "were not significant" (Centra, 
1972:18). 

Administration of Student Rating Instruments 

Procedures used in administering the student rating instrument 
could influenca ratings made by students and the usefulness of results 
themselves. For this reason, it is generally agreed that instructor 
influence should be eliminated from procedures to the extent possible. 
Seve: al factors were examined for this study. They were: "(1) anonym- 
ity of stud-snts, (2) presence of the instructor, (3) systematic nature 
of procedures, (4) timing of administration, (5) type of instrument, 
and (5) voluntariness of instructor. 

There appears to be a general feeling that people should be 
accountable for their actions. Rating courses and instructors are 
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actions of students, so it can be expected that they identify 
themselves when completing the » ating instrument. It is also believed 
that fear of instructor retaliation will result in inflated ratings if 
students need to identify themselves. This results in a dilemma. 
Centra (1980:44) reported results of two studies conducted to determine 
the extent and direction of the influence of having or not having 
students identify their ratings. He reported that Stone, Rabinowitz, 
and Steel (1977) concluded, "Although students who identify themselves 
are expected to be far more generous in their ratings . . . evidence 
does not totally support this expectation." He also reported on a 
companion study conducted by the same researchers durinr the same year 
after which they concluded that expected higher ratings were found when 
students signed the form. Schrader (1986:3-4) reported that only 
5 percent of the students indicated they would have rated the instruc- 
tor differently had the student's name been required. The general 
conclusion re .hed as a result of revi^wimi reasons for their answer 
was that students were interested in providing accurate data to help 
instructors improve. It is possible that some students may rate an 
instructor higher because they feel reprisal for lower ratings from the 
instructor in future courses, recommendations, etc. It seems that most 
students, however, did not have that fear. 

The extent to which an instructor's presence influences stu- 
dent overall ratings of the instructor has been studied, and there 
seems to be some level of influence. Instructor intimidation was 
suggested as a reason for inflated rating levels that resulted. Miller 
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(1974:29) reported results of a study conduct<id at the University of 
Kentucky in which he concluded, "Student ratings were statistically 
higher--significantly so—when the instructor was present," his would 
support recommendations that someone other than the instructor should 
administer the rating instrument and that the instructor should not be 
in the same room while students are completing the rating instrument. 

In order for student rating instruments to be used with maxi- 
mum effectiveness, the instruments should be administered and collected 
in a responsive and systematic manner (O'Hanlon and Mortensen, 
1980:669). Some conditions to be addressed are when in the semester 
the instrument is administered, instructions to the student for 
completing the instrument, detail and procedures for administration and 
collection of the instruments, and time available for completion of the 
instrument. For data .o be useful in identifying the individual's 
professional growth goals and evaluating progress toward meeting those 
goals, student ratings on similar elements is also necessary. 

While open-ended questions need to be a part of the evaluation 
(O'Hanlon and Mortensen, 1986:665), they would probably not provide the 
instructor with sufficient numbers of students indicating the same 
concerns or levels of concerns on which to base the growth program. In 
addition, an educator's interpretation of the remarks may not be the 
same as the meaning intended by the student. This difference potential 
makes this type of questionnaire less desirable. 

Another factor is concern for practicality with this type of 
instrument when large numbers of students are involved. Most colleges 
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collect data for evaluating instructors from students; and because 
evaluations are collected from all students in many or all courses, "it 
has become common to use standard or semi standard rating forms" 
(O'Hanlon and Mortensen, 1980:664). 

The timing l.' the administration of the rating instrument has 
been a concern of a number of researchers as well, it was recommended 
that ratings be based on sufficient instructor contact to provide the 
student with a sound basis for making judgments. It would seem that by 
midsemester, sufficient time has elapsed to enable the student to do 
accurate ratings. Aleamoni (1978-296) reposed the results of two 
studies by Miller (1971) .nd Centra {lS7i) in which the same conclusion 
was reached. "No significant differences were found between the mid- 
semester and end of semester student ratings of instructors." These 
support the recoiranendation that instruments bs administered at some 
time between midsemester and the end of the semester. 

The procedure for selecting courses for administration of 
rating instruments was also of interest. To what extent can the 
instructor bias the results by volunteering or not volunteering courses 
for administration of these student rating instruments has been a 
question. Voluntariness could be expected to positively influence the 
ratings given the instructor as only those courses that were expected 
to produce high ratings would be volunteered. A study conducted by 
Cashin and Kerrin (1983:595) provided results which did not support 
this basis of influence while they candidly admitted that some differ- 
ences occurred. They concluded that "none of the differences was of 
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practical significance, " There will probably be no effect on the 
ratings if the instructor volunteers to have students in courses 
complete the ratings or if a supervisor of the instructor or other 
administrative person directs to have students in courses taught by the 
instructor complete the rating instrument. 

Use of Student Ratings in Identifying Individual 
Professional Growth Goals 

Widespread agreement was found that the most important purpose 
of evaluation is to provide data that will enable the individual 
instructor to identify professional growth goals. The instructor has 
the "major responsibility for the identification of learning objec- 
tives" (goals) for the development program (McLean, 1985:5). In the 
process of identifying these goals, she emphatically states that the 
instructor must receive feedback on performance. This feedback is 
provided by the student ratings data on eacn of the various elements, 
not just the overall instructor rating alone. The importance of stu- 
dent ratings and improvement of student achievement is illustrated by a 
study reported by Aleamoni (1984:140-141), "When the student ratings 
were used to identify goals and activities completed in accordance with 
the goals, they not only rec ived higher ratings at the end of the 
year; but their students also scored higher on achievement tests." 
Altshuler and Richter (1985:59) emphasized that "teachers need to know 
how they are perceived and valued" in order to develop their goals. 
The student ratings are an "indication of their teaching success . . . 
the results to shape their subsequent pedological behavior" (Bausell 
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and Magoon, 1972:1013). Biehler (1984:41) also emphasized that 
instructors should "take seriously the opinion of students" whtn 
developing their goals for professional growth. Instructors interested 
in improving their instructional skill identified goals to assist them 
in determining th^ growth to be pursued. Braskamp, Brandenburg, and 
Ory (1984:20) emphasized that in order to imprc^'e instruction, the 
instructors need to "specify goals and receive feedback about their 
progress towards achieving those goals." They (1984:24) also reminded 
educational leaders and staff development special ists that it is 
i iportant that student course and instructor characteristics [elements] 
also be taken into account when iniormation is interpreted for assess- 
ing teaching competence. They (1984:25) then reiterated, "Evaluation 
of professional development [growth] and improvement of instruction are 
inseparable." 

Millar (1974:4) indicated that this is also an important 
consideration because these elements are "dependent on local issues." 
Types of students, types of programs, and types of courses are unique 
to each institution. Experience with the extent of influences of each 
of the elements is important when using the student ratings as the 
basis for identifying the individual's professional growth goals. 
Murray (1983 : 138) al so emphasized the importance of the 1 ocal 
considerations in "improving the diagnostic and remedial value of 
student instructional ratings." 

The more known about local college issues, the more realistic 
goals will be. ihis should lead to increased student learning as a 
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result of the instructor participation in the growth program. 
Instructors continue participating in a growth program that results in 
iticreased student achievement and increased levels of student ratings 
because they see a payback for the time and effort invested. Student 
ratings data can then be used to help instructors determine where their 
strengths and weaknesses lie (Cohen, 1980:323). Harrell (1980:5) 
supported this idea with almost identical words regarding faculty 
strengths and weaknesses. He further emphasized the importance of the 
student ratings in orofessional growth as he states, they become t, a 
"principal basis fo>' faculty selection of an activity or group of 
activities for faculty development." 

Use of student ratings cannot be construed as the solution to 
all teaching problems. They only indicate what students perceive to be 
the instructor's strengths and particular weaknesses related to less 
than effective student learning (Hunter, 1982:7). The strengths can 
then be further developed and the weaknesses addressed in the 
professional growth program. 

Summary 

In summary, review of the literature regarding elements indi- 
cates that potential biasing elements have been the subjects of many 
studies and articles. Conflicting results have been reported. 
Therefore, it is difficult, if not impossible, to state that an element 
will or will not affect the student ratings of instructors. 
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Even when significant correlations were found between e"!aments 
and the students' overall ratings of instructors, in many instances an 
explanation supported a corresponding relationship rather than a 
biasing or cause-effect relationship. Fur several elements (e.g., 
feedback and grading practices), it appeared that it was not the 
element itself but rather how the element was presented to students 
that caused the influence. This supported the need for delving into 
the how of an element; i.e., being more diagnostic o*"iented. 

Variations in definitions, procedures, etc., also make it 
somewhat difficult to predict the extent of an element's influence on 
student overall ratings of instructors in a particular situation. The 
reviewed literature supported the need for studies in which the 
influence of each element oi. student overall ratings of instructors are 
addressed at the local postsecondary institution. 

A number of elements were identifitd in the literature as 
having a significant correlation or a generally positive trend with the 
students overall ratings of instructors. The following were included: 
(1) explicit explanation of grading practice, (2) good objectives and 
content agreement, (3) good organization of instructional materials and 
presentations, (4) good actual and expected grade agreement, 
(5) exhibited instructor interest in the student, (5) exhibited 
Instructor opennass to views of others;, (7) effective instructor 
presentations, (8) compatible instructor personality, and (9) exhibited 
instructor respect for students. 
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The element (class size) appeared to have a negative Influence 
on the overall ratings of Instructors. However, even though It may 
Influence the overall ratings of Instructors, the significant differ- 
ence occurs only when comparl ng 1 nstructors teachi ng courses (or 
sections of courses) having considerable differences in the number of 
students. In most Institutions, great differences are exceptions, not 
the rule. 

Several results supported the poi;s1b111ty that Instructors 
could manipulate an element over a short tlirie period as was Illustrated 
by the Doctor Fox Effect (Naftulln, Ware, and Donnally, 1973:534). 
However, most results i .dicated the majority of students were not 
Influenced to any ^r^at extent by short-term efforts. Most student 
judgments seem to be based on the long-term effects of the element. 

Results of the potential biasing effect of the twenty-four 
elements Included In this section as reported In the literature were 
not sufficiently conclusive to transport their effects to student 
ratings at other colleges. Findings In the literature did, however, 
support the use of student ratings as an Important component of the 
data base on which Instructor professional growth goals could be 
developed. The results also supported further study of the biasing 
Influences of elements on student ratings of Instructors at Individual 
colleges. Adjusting student ratings in accordance with known biasing 
effects of el ements wi 1 1 then be poss 1 bl e . Thi s will er.abi e the 
college to obtain data having Increased validity for decisions regard- 
ing instructor growth goals. 



CHAPTER 3 
PROCEDURES AND METHODOLOGY 



The purpose of this study was to refine a part of the instruc- 
tional assessment procedures at LTC. This was accomplished by obtain- 
ing and analyzing data for course, instructor, and student elements to 
determine their relative relationships to student overall ratings of 
instructors. Equally important, this study also determined the biasing 
effects of particular course, instructor, and student elements; student 
demographics; and college course ard program classifications on student 
overall ratings of instructors. The data were student ratings of 
particular instructional elements and student overall ratings of 
instructors in courses taught during the 1986-87 fall semester at the 
two-year postsecondary institution, Lakeshore Technical Institute, now 
named Lakeshore Technical College. The Student Assessment of Instruc- 
tion (see Appendix H) instrument was used for obtaining data. 

The study was designed to address the relationship of the 
various course, instructor, and student elements on students' overall 
assessment of instructors for selected courses in occupational 
programs. Included in tins chapter are the following three sections: 
(1) Research Questions and Hypotheses, (2) Procedures for Collecting 
the Data, and (3) Procedures for Treating the Data. 
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Research Questions and Hypotheses 

Research Questions 

This study was undertaken to answer the research questions 
included in Chapter 1. The research questions were: 

1. Is there a relationship between student overall ratings of 
instructors and student ratings of elements in the three major groups 
(course, instructor, and student) in the assessment of instruction at 
LTC? If so, what is the nature of the relationship? 

2. Is there a relationship between student overall ratings of 
instructors and student ratings of particular items within each of the 
three major element groups, student demographics, or place of the 
course in college academic programming? 

Research Hypotheses 

These questions were rewritten so that statistical test data 
analyses might provide answers directly. The research questions were 
rephrased into the following statistical research null hypotheses: 

H{01): Differences in student overall ratings of instructors 
are related to different ratings that students give to the elements, as 
an aggregated measure in the rating instrument, that pertain to the 
(a) course, (b) instructor, and (c) student. The aggregated measure 
for each element is the composite of the separate items pertaining to 
the indicated elements. 
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H{02): Differences in the student overall ratings of 
instructors are related to the ratings students give, to each of the 
items on the rating instrument that pertain to the first element in 
research question one (the course). 

H{03): Differences in the student overall ratings of instruc- 
tors are related to the ratings students give to each of the items on 
the rating instrument that pertain to the second element in research 
question one (the instructor). 

H(04): Differences in the student overall ratings of instruc- 
tors are related to the ratings students give to each of the items on 
the rating instrument that pertain to the third element in research 
question one (the student). 

H(05): There is a significant difference between the overall 
ratings of instructors by students in courses having one function and 
by students in courses having another. 

H{05): There is a significant difference between the overall 
ratings of instructors by students in one nurse level and by students 
in another. 

H{07): There is a significant difference between the overall 
ratings of instructors by students ^n one age group and by students in 
another. 

H{08): There is a significant difference between the overall 
ratings of instructors by students of one sex and by those of the 
other. 
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H{09): There is a significant difference between the overall 
ratings of instructors by students from one division and by students 
from another. 

H{10): There is a significant difference between the overall 
ratings of instructors by students expecting one grade in the course 
and by students expecting another. 

H{11): There is a significant difference between the overall 
ratings of instructors by students exerting one level of out-of-class 
preparation and by students exerting another level. 

H{12): There is a significant difference between the overall 
ratings of instructors by students feeling the course had one level of 
difficulty and by students feeling the course had another level. 

These null hypotheses were tested at the .05 level of signifi- 
cance using a two-tailed test. Multiple Regression Analysis and One- 
Way Analysis of Variance statistical techniques were used to test the 
hypotheses. 

Procedures for Collecting the Data 

Population and Sample Used in the Study 

The college offers fifty associate degree and vocational 
diploma programs. Approximately 800 students were enrolled in these 
programs during the fall semester of 1986-87. In order to develop a 
sample of this population, students in eighteen programs were selected 
for t*"* study according to the following criteria: 
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1. Student ratings of the instructors of both selected 
courses in the program could be obtained. The college does not require 
student assessments in all courses offered. 

2. Students were in a program designed to be completed on a 
full-time basis in one or two years. Students in programs designed to 
ba completed on a part-time basis in five years were not included. 

3. Students were in a program that included both a general 
education course ard an occupational -specific course during the 1986-87 
fall semester. A number of programs did not include a ge-;eral 
education course during the fall semester. Students in these programs 
were not included in the study. 

The population consisted of 806 students enrolled in the first 
year of full-time occupational associate degree or vocational diploma 
programs having both an occupational -specific and a general education 
course offered during the fall semester of the 1986-87 school year. 
From this population, a sample was drawn by using all students in the 
first occupational-specific course and the first general education 
course listed in each occupational program selected for inclusion in 
the study. Student ratings for the second course listed in the program 
sequence file were substituted if ratings could not be obtained for the 
first course in each function group. Both the first course and the 
substitute course included the same students. If student ratings could 
not be obtained from either the first course or the substitute course 
in both function groups, students from the particular program were 
excluded from the study. Most programs had one or only several 
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occupational -specific courses and one general education course offered 
during the fall semester. The sample included 262 students enrolled in 
eighteen programs selected for inclusion in the study (see Appendix G). 
This number represented 33 percent of the students enrolled in first or 
second semester courses of the first year of full-time occupational 
program* during the 1986-87 fall semester. 

Of the possible 524 (262 students each completing ratings in 
two courses) instruments, 499 completed instruments were obtained (see 
Table 3.1). Of this number, 457 (87 percent) were usable. Specific 
ratings or information needed for the study was missing on forty-two 
instruments. Students were not told beforehand when the instrument was 
to be administered. Student absences on the date scheduled for 
administration of the assessment instrument were judged to be normal by 
the instructors. No pattern of missing ratings or supplemental 
information on the assessment instrument was observed that could be 
attrif'uted to gender of student, cou»*se function, or program level. 
Therefore, it was concluded that there was no difference between those 
students not present or not completing the entire instrument and those 
students present and completing the entire instrument. Because of 
anonymity used in administering the instruirsnt and missing supplemental 
information on some assessment intruments, the demographics of some 
students could not be determined. 

Instrument Used in the Study 

Thu Student Assessment of Instruction instrument (see Appen- 
dix E) developed by members of an LTI quality circle group during the 
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1984-85 and 1985-86 school years was used for collecting data. The 
development of the instrument was not a part of the MARP herein 
reported. The Student Assessment of Instruction instrument consisted 
of three groups of items. These items represented particular 
(1) course elements, (2) instructor elements, and (3) student elements. 
One item requiring an overall assessment of the instructor was also 
included. In addition, information on program, course function and 
level, and student age and gender was requested. The student's name 
was not requested. 

Table 3,1 

Number of Students and Instruments Used in the Study 



Item Number 

Number of students enrolled in eighteen programs included 

in the study 262 

Number of completed assessment instruments expected 

(number of students enrolled times two courses per student) 524 

Number of completed assessment in:truments received 499 

Number of usable assessment instruments received 457 

Percent sample 33 
students in sample of total students (262 806 x 100) 

Percent usable replies 87 



number of usable instruments number of students enrolled x 100 
2 

(499 ^ 262 X 100) 
2 
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Each item had five choices with gradations similar to a Likert 
scale except that each of the choices was labeled with "A" being the 
most positive. The choices varied as were appropriate for the item. 
The student was to select the one that must agreed with the opinion of 
the student. The student recorded each rhoice on the instrument. 

The validity of che instrument was addressed during its 
development m 1984-86. This was accomplished by comparing the items 
Included t? those items included on other commercially available forms 
and to those included in a number of research studies. The instrument 
was also submitted to the entire instructional staff of approximately 
120 instructors, managers, and specialists for reactions and comments. 
Staff members were ir ormed that the instrument for collecting data was 
to be us d in identifying goals for instructor professional growth and 
that they were to react to it and make suggestions accordingly. The 
circle reviewed the staff comments and suggestions and made decisions 
regarding the addition or deletion of items and modifications in 
wording as necessary to improve the wording of each item after each of 
the three administrations. 

The reliability of the instrument was addressed by administer- 
ing the instrument to students in selected courses at the end of the 
summer session and then again at the end of the first and second 
semester of the l!;35-86 school year. Revisions were made, based on 
instructor and student written comments, after each administration. A 
revised instrument was used for each succeeding term. The sample of 
courses selected included day and evening courses, lecture and 
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labovatory courses, single-instructor-taught and team-taugnt courses, 
and occupational and general education courses. 

When the Student Assessment of Instruction instrument was 
first administered, students were requested to select the one choice 
that best fit their feelings regarding the item and to further clarify 
the choice through the inclusion of a written comment, if desired. In 
addition, students were to include comments about the wording, choices, 
and/or difficulties encountered in selecting a choice for each item. 
Item choices and comments were reviewed by circle members and 
modifications made after each administration. Approximately 400 
students were involved in this process. 

Tht: actual reliability of the assessment instrument was deter- 
mined to obtain a measure of the consistency of student responses in 
this study. The reliability test used was the Equal Spearman-Brown 
Reliability component of the Split Model (SPSS Inc., 1986:860). This 
reliability test was selected because the instrument contained thirty 
items used in the study, fifteen in each half; item number one was not 
used because the courses were required. The test yielded a reliability 
coefficient of +0.82. 

Administration of the Instrument 

Instructions for administering the assessment instrument were 
developed (see Appendix H). These detailed directions were to be 
followed by each of the four division assistants in administering the 
instrument. Information on the importance and use of *-he data obtained 
was also provided to the students. 



103 



95 



The instrument was scheduled to be administered at the 
beginning the class period. The class period selected by the 
instructor was convenient and interfered least with other instructional 
activities. The instrument was administered in selected courses 
between the thirteenth and seventeeiith week of the semester. 

The instrument was also administered to students in a number 
of courses not selected to be a part of the study. This was done 
because either the instructor desired student as.',essments in additional 
courses or sections of a course than the one{s) included in the study 
or desired student assessments in courses because the instructor had 
not taught a course or section of a course included in the study. 
Although the data for these additional courses or sections of courses 
were collected, tabulated, analyzed, and returned to instructors for 
use in discussions with associate administrators, the data were not 
included in the study. 

Students responded on a duolicated copy of a preliminary 
Scantron Corporation printed machine-scorable instrument. This change 
in the procedure was required because modifications during the develop- 
ment of the paste-up resulted in a delay in the actual printing of the 
instrument. Therefore, commercially printed copies did not arrive 
until after the scheduled administrations of the instrument. This 
delay and use of duplicated copies necessitated a transfer of student 
responses to printed instruments by research office staff, once they 
arrived. An edit check was performed to ensure accuracy of the 
transfer. 
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The instrument was administered to students enrolled In 
general education and occupational courses In eighteen programs. These 
programs were: 

1. Administrative Assistant—Information Processing. 

2. Administrative Ass1sstant--Secretar1al . 

3. Associate Degree Nursing. 

4. Child Care Services. 

5. Court and Conference Reporting. 

6. Data Processing. 

7. Dispensing Optician. 

8. ».lectrical Power Engineering Technician. 

9. Electromechanical Technology. 

10. Electronics Technician. 

11. Graphic Arts 

12. Marketing. 

13. Materials Management. 

14. Mechanical Design Technician. 
15 Office Assistant. 

16. Pharmacy Technical Aide. 

17. Plastics Technician. 

18. Word Processing Specialist. 

Of the twenty-four courses selected for the study, eight were 
classified as general education function and sixteen as occupational- 
specific function. Because a number of courses were classified as the 
same furivtion for more than one program, less than thirty-six courses 
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were selected. In some cases, students were in separate sections of a 
course according to the particular program. In othero, students from 
nore than one program were in the same section of a course. The 
courses included in the study are listed in Table 3.2. The selected 
courses in each program are listed in Appendix G. 

Lnrollment data indicated an enrollment of 262 students in the 
first semester of the eighteen programs. For various reasons, not all 
students were enrolled in both selected courses. Only instruments com- 
pleted by students that indicated programs selected for inclusion in 
the study were used. Students from other programs were in course 
sections and completed the instruments following the same directions as 
the students in the study. Their instruments were eliminated from the 
study. A total of 499 instruments were collected frum students in the 
study. 

Variables Used in the Study 

A number of variables were identified as important in this 
study. The independent variables used in the study were the following 
course, instructor, and student elements; student demographics; and 
college course and program classifications. The data was obtained from 
student ratings of items or supplemental information provided by the 
student on the instrument as requested. 

Course Element Variables . 

1. Assignments-objective agreement level (Item 10). 

2. Bias level (Item 14). 
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Table 3.2 
Courses Included 1n the Study 



General Education Courses 



Occupational -Specific Courses 



801-190, Business and 
Professional Spetch 

801-356, Communications 

801-353, Communications Basic 

801-366, Communications Graphics I 

801-155, Communications Office I 

801-151, Communication Skills I 

801-110, Economics 

809-164, Human Growth and 
Development 



307-303, Art Activities 

620-140, Basic Electricity 

101-331, Clerical Accounting 

107-120, Data Processing 
Concepts 

606-105, Dimensions and 
Working Drawings 

605-105, Direct Current 
Fundamentals 

536-322, Drug Classification 

105- 383, Electronic Word 
Processing 

106- 164, Information Process- 
ing Concepts 

619-110, Introduction to 
Plastics 

209-305, Lithography Theory 

106-168, Machine Shorthand I 

104-102, Marketing Principles 

510-110, Nursing I 

516-104, Ophthalmic Optics 

194-178, Overview of Marketing 
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3. Evaluation-objective agreement level (Item 11). 

4. Information usability level (Item 7). 

5. Instructional materials usability level (Item 13). 

6. Planninc/organizational level (Item 8). 

7. Subject matter-objectives agreement level (Item 9). 

8. Text value level (Item 12). 

Instructor Element Variables . 

1. Class session organization level (Item 20). 

2. Comments on assignments level (Item 27). 

3. Course related skills demonstration lev.-l (Item 18). 

4. Enthusiasm level (Item 19). 

5. Explanation of general course information (Item 15). 
5. Interest in student learning level (Item 28). 

7. Matferial explanation level (Item 17). 

8. Practical application of course level (Item 29). 

9. Problem solving/thinking encouragement level (Item 22). 

10. Promptness of assignments return level (Item 25). 

11. Respect f . r student level (Item 30). 

12. Speech appropriateness level (Item 23). 

13. Student participation encouragement level (Item 21). 
Subject knowledge level (Item 15). 

15, Teaching aid use (Item 25). 

15. Written communications clarity level (Item 24). 



113 



100 



student Element Variables . 
1- Difficulty level (Item 4). 
Z. Expected grade (Item 6). 

3- Opinion of course— initial level (Item 2), 

4- Opinion of course—present level (Item 3), 
5. Expected grade (Item 6). 

Student Demographic Variables . 

1. Student age (supplemental information provided by the 
student on the assessment instrument). 

2. Student gender (supplemental information provided by the 
student on the assessment instrument). 

College Course and Program Classification Variables . 

1. Course function (supplemental information provided by the 
student on the assessment instrument). 

2. Course level (supplemental information provided by the 
student on the assessment instrument). 

3. Instructional division (supplemental information provided 
by the student on the assessment instrument). 

The course, instructor, and student variables were used in 
answering research question one. The student element, student 
demographics, and college course and program classification variables 
were used in answering research question two; The instructional 
administrators believed that the course and instructor element 
variables were not appropriate for answering research question two. 
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The dependent variable used was the student overall rating of 
the instructor (Item 31). This denendent variable was used to answer 
both research questions one and two. 

Procedures for Treating the Data 

The Student Assessment of Instruction Instrument, completed by 
each student, had thirty-one items including thirty course, instructor, 
and student .element items and one overall student assessment of the 
instructor item. Additional supplemental information blanks were pro- 
vided for course number, course function and level, program designa- 
tion, student age, and student gender. The data for items representing 
eight course elements, sixteen instructor elements, and five student 
elements and supplemental information included on the assessment 
instrument were used in the study. One item (my reason for taking the 
course) was not included in the study because all courses included in 
the study were required for a degree or diploma. 

Creating the Data File 

The first step in tabulating and computer processing data was 
the development of a software program that would capture student data. 
LTC instructional computer specialist, Frederick Crook, developed the 
program with assistance from staff at the Scantron Corporation office 
in California. Using the developed software program, the data entry 
terminal was able to capture the student ratings and supplemental 
information and load them on an IBM PC disk in a machine language 
format . 

ERIC 1V6 
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The next step was to upload the data from the IBM PC disk to 
the Data General computer. The LTC computer specialist developed a 
program to (1) upload the data into assigned files, (2) change the 
machine language format to an alpha-numeri ; format, and (3) enable 
entry into a file to permit editing (i.e., correct obvious errors in 
supplemental information, correct omissions, etc.). This program ^as 
then used to upload data from the IBM PC disk and to edit the data in 
the files. 

Because of storage capacity, limitations of the IBM PC disk, 
the size of the sample, and the number of bits of information to be 
captured on each instrument, the LTC computer specialist developed a 
software program able to upload data from a number of IBM PC disks and 
combine or group files after being uploaded. This was then done so 
that various files could be combined; and for each independent 
variable, values determined and analyses completed. 

Conducting the b .tistical Analyses 

The data were submitted to the SPSS'^ Multiple Regression 
Analysis (SPSS Inc., 1986:663-686) to answer the first research ques- 
tion, "Is there a relationship between student overall ratings of 
instructors and student ratings of items in the three major element 
groups (course, instructor, and student) in the assessment of 
instruction at LTC? If so, what is the nature of the relationship?" A 
second multiple regression analysis was made to determine the 
relationship of the student ratings of each course element with student 
overall ratings of the instructors. A third multiple regression 
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analysis was made to determine the relationship of student ratings of 
each instructor element with student overall ratings of instructors.. A 
fourth multiple regression analysis was made to determine the 
relationship of student ratings of each student element with student 
overall ratings of instructors. The significance was tested at the .05 
level using a two-tailed test. 

The constant (k) and correlation coefficient (B) for each 
element were determined. The multiple regression analysis test compu- 
tations were conducted until the .05 level of significance was met for 
each group. The multiple regression equation for determining the value 
of Item 31 (Q31) from the values found for each of the significant ele- 
ments was then developed. 

Xq3j = k + B.X. + . . . + Bj.X. 

where Xq^j = student overall instructor rating 

k » constant 

B = regression coefficient for element 
X = student rating 

i; j = elements found to have a significant relationship with 
the Q31 value 

The independent variables used for answering this question 
were levels of a student's rating on course items, instructor items, 
and student i jms on the student assessment instrument. Each had a 
possible student rating of one through five. The independent variables 
are listed below. 



er|c 



104 



Course Element Variables . 

1. Assignments-objective agreement. 

2. Bias. 

3 . Eval uati on-object i ve agreement . 

4. Information usability. 

5. Instructional materials usability. 

6 . PI ann i ng/organi zati on . 

7. Subject matter-objectives agreement. 

8. Text value. 

Instructor Element Variables . 

1. Class session organization. 

2. Comments on assignments; 

3. Course related skills demonstration. 

4. Enthusiasm. 

5. Explanation of general course information. 

6. Interest in student learning. 

7. Material explanation. 

8. Practical application of course. 

9. Problem solving/thinking encouragement. 

10. Promptness of assignments return. 

11. Respect for student. 

12. Speech appropriateness. 

13. Student participation encouragement. 

14. Subject knowledge. 
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15. Teaching aid use. 

15. Written coinmuni cat ions clarity. 

Student Element Variables . 

1. Difficulty. 

2. Expected grade. 

3. Opinion of course--initial . 

4. Opinion of course--present. 

The data were submitted to the One-Way Analysis of Variance 
(ANOV;.) (SPSS Inc., 1986:465-474) to answer the second question, "Is 
there a relationship between student overall ratings of instructors and 
student ratings of particular items within each of the three major 
element groups, student demographics, or place of the course in college 
academic programming?" The data were tabulated and, where appropriate, 
means and standard deviations calculated using the LTC Data General 
computer and the Statistical Package for the Social Sciences'^ (SPSS'^) 
software package for each of the supplemental data items and course, 
instructor, and student elements. The F ratio was obtained for the 
groups withi.i the variable to determine if the student rating of the 
instructor {Q31) means for the various groups were significantly 
different. 

The groups selected for each of the Independent variables 
Included the following: 
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Student Demographic Variables . 

1. Age (sixteen to twenty-two, twenty-three to twenty-nine, 
thirty plus years of age). 

2. Gender (female, male). 

College Course and Program Classification Variables . 

1. Course function (occupational, t^eneral education). 

2. Course and program level (associate degree, vocational 
diploma). 

3. Instructional division (Business and Marketing, Health 
Occupations, Home Economics, Trade and Industry). 

Student Elements . 

1. Expected grade (A, B, C, I, don't know). 

2. Expression of course difficulty (very difficult, diffi- 
cult, average, easy, and very easy). 

3. Opinion of course-- initial (highly positive, positive, 
neutral, negative, very negative). 

4. Opinion of course--preserit (highly positive, positive, 
neutral, negative, very negative). 

5. Out-of-class preparation (fifteen plus hours per week, ten 
to fifteen hours per week, five to ten hours per week, one to five 
hours per week, zero hours per week) . 
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CHAPTER 4 
PRESENTATION OF THE RESULTS 



This chapter includes a presentation of the results. Chapter 
Four includes the following four sections: (1) Tabulations, Means, and 
Standard Deviations of Student Ratings, (2) Determinations of Relation- 
ships Between Items and Student Overall Ratings of Instructors, 
(3) Correlations Between Ratings of Items and Student Overall Ratings 
of Instructors, and (4) Comparison of Student Overall Ratings of 
Instructors by Groups. 

Tabulations, Means, and Standard 
Deviations of Student Ratings 

Student ratings as recorded on the assessment instrument were 
tabulated, and the mean and standard deviation of their ratings were 
calculated for those items pertinent to each of the major elements of 
interest in this study. The mean and standard deviation for each item 
are included in Table 4.1 and Appendix I. Item number one (reason for 
taking the course) was not included in the study because all students 
in the programs selected were enrolled in required courses in their 
specific program. 

Means of student ratings for the it-.ds as indicated in 
Table 4.1 ranged from a high of 4.79 to a low of 2.54. Means of ten 
items were greater than 4.50. Means of twenty-two items were greater 
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than 4,00, Only the mean for one item, Q5 (weekly out-of-class prepa- 
ration), was less than 3,00- 

Table 4,1 

Ranking of Instrument Items by Means of Student Ratings 



1 * 2 ^ Standard 

Rank Item Mean'^ Deviation 

1 Q16, instructor knew subject 4-79 0,43 

2 Q14, course free from bias 4,75 0.61 

3 Q15 instructor explanation of written 

course information 4,69 0,78 

4 Q19, instructor showed enthusiasm 4,66 0,61 

5 Q30, instructor showed respect 4,66 0,67 

6 QIO, assignment agreement with course 

objectives 4,63 0.64 

7 Qll, evaluation agreement with course 

objectives 4,60 0.66 

8 Q26, timely return of assignments 4,57 0.73 

9 Q18, instructor demonstrated skills 4,52 0,70 

10 Q9, subject matter agreement with 4,51 0,68 
objectives 

11 Q21, instructor encouraged student 

participation 4,48 0.81 

12 Q22, instructor encouraged thinking 4,43 0.73 

13 Q20, instructor organized classes/labs 4,40 0.78 

14 Q17, instructor explained material clearly 4,35 0,75 

15 Q24, instructor communicated in wriiing 4,33 0.90 
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Table 4.1 Continued 



„ , 1 y , Standc-.rd 

Rank' Item'^ Maan^ - • •• 



Maan Levitt ion 
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16 Q28, instructor interested in r^y learning 4.28 0.85 

17 Q23, instructor spoke in a way to help me 

learn 4.27 0.90 

18 Q7, information regarding procedures was 

usable 4.20 0.74 

19 Q25, instructor use of AVs 17 LOO 

20 Q27, instructor comments regarding my 

assignments 4.11 1.02 

21 »/29, instructor application of course to 

life or work 4. 06 0.99 

22 Q8, course planning and organization 3.99 0.83 

23 Q12, textbook{s) value in unaerstanding 

course 3.98 1.02 

24 Q3. present opinion of course 3.93 0.85 

25 Q6, expected grade 3.82 1.13 

26 Q2, initial opinion of course 3.64 0.94 

27 Q13, instructor ma*erial{s) value in 

understanding course 2.57 1.38 

28 Q4i course difficulty 3.32 0.84 

29 Q5, weekly out-of-class preparation 2.64 0.90 



^Rank with 1 being highest 
2 

Item wording shortened for table; the full text is included in 
Appendix F 

3 

Mean is based on a 5-point scale with 5 being most positive 
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standard deviations for stur' it ratings of items ranged from a 
high of 1.38 to a low of 0.43. Thirteen had values between 0.60 and 
0.79. Only five had values greater than 1.00. 

Two items having the greatest means, Q16 (the instructor knew 
the subject well --4. 79), and Q14 (course content and materials were 
free from bias--4.75), also had the smallest standard deviations, 0.43 
and 0.61 respectively. The item Q31 (student overall rating of the 
instructor) had a mean of 4.15 and a standard deviation of 0.73 (see 
Appendix I). This mean was similar to the mean for the item ranked 
number twenty of twenty-nine items included in the study. 

Determination of Relationships Between Items and 
Student Overall Ratings of Instructor! 

Data were submitted to the SPSS^ multiple regression analysis. 
This analysis was used to determine if there was a significant differ- 
ence between the proportion of variance in the student overall ratings 
of instructors explained by student ratings to any particular course, 
instructor, or student item and th?t explained by student ratings to 
any other particular course, instructor, or student item. The level of 
sic-^ficance selected was the .05 level with a two-tailed test. The 
null hypothesis developed was as follows: 

Null Hypothesis H(01): There is no significant difference 
between the proportion of variance in the student overall instructor 
ratings explained by student ratings of one of the course, instructor, 
or student elements as compared to each of the others. 

H(01): u(2) = u(3) - u(4) > . . . = u(30). 
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Results of the multiple regression analysis are included in 
Table 4.2. The calculated F was 114.03, which was significant at less 
than the .0001 level. Therefore, the null hypothesis was rejected, and 
the opposite established. There is a significant difference between 
the proportion of variance in the student overall ratings of instruc- 
tors explained by student ratings of one of the course, instructor, or 
student items as compared to each of the others. 

Further analysis revealed that student ratings to five items 
explained 95 percent of the variance in student overall ratings of 
instructors. These five items were Q23 (instructor spoke in a way that 
helped me learn), Q8 (the course was well planned and organized), Q22 
(the instructor encouraged thinking, problem solving, and decision 
making), Q17 (the instructor explained materials clearly), and Q28 (the 
instructor demonstrated an interest in my learning). 

A regression formula was developed to determine the expected 
value (X) of item Q31 (student overall ratings of the instructor) if 
student ratings of significant course, instructor, and student items 
were known. The "8" coefficients are included in Table 4.2. The 
regression formula developed was as follows: 

Xp3j = 0.64 + 0.17X(j23 + 0.26X(jg + 0.16X^22 + °-l*^Q17 + 
0.10X^28. 

Data were submitted to the multiple regression analysis to 
determine the significance of differences between the proportion of 
variance in student overall ratings of instructors explained by student 
ratings of any particular student item as compared to any other student 
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Table 4.2 

Determination of Significant Student Ratings of Course, 
Instructor, and Student Items That Explain 
Student Overall Ratings of Instructors 

Analysis o. Variance 



Degrees of Freedom Sum of Squares Mean Squares 



Regression 5 
Residual 49:) 
F = 114.03* 

Probability < 0.0001 level 
*F is significant 



144.03 
124.55 



28.81 
0.25 



Significance of Items From All Elements Covered 



Item^ 

Q23, instructor spoke in a way to 
help me learn 

Q8, course planning and organization 

Q22, instructor encouraged thinking 

Q17, instructor explained material 
clearly 

Q28, instructor interested in my 
learning 

k, constant 

p < 0.05 level with two-tailed test reached 



Constant 



0.64 



6 

0.17 
0.25 
0.16 

0.14 

0.10 



3.94 
7.54 
4.25 

2.98 

2.86 
3.89 



Item wording has been shortened for table; the full text is included 
in Appendix F 
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Item. The level of significance selected was the .05 level with a two- 
tailed test. The null hypoth«iSis developed was as follows: 

Null Hypothesis H{02): There is no significant difference 
between the proportion of variance in the student overall ratings of 
instructors explained by student ratings of one of the student elements 
as compared to each of the other student elements. 

H{02): u{2) - u{3) - u{4) » u{5) = u{6). 

Results of the multiple regression analysis are included in 
Table 4.3. The calculated F was 141.80. which was significant at less 
than the .0001 level. Therefore, the null hypothesis H{02) was 
rejected and the opposite established. There is a significant 
difference between the proportion of variance in the student overall 
ratings of instructors explained by student ratings of one of the 
student elements as compared to each of the other student elements. 

Further analysis revealed that student ratings to one student 
item explained 95 percent of the variance in student overall ratings of 
instructors. That student item was Q3 (my opinion of the course at 
present is). 

A regression formula was developed to determine the expected 
value of Q31 (student overall rating of the instructor) if student 
ratings of significant student items were known. The "B" coefficient 
is included in Table 4.3. The regression formula developed was the 
following: 




- 2.56 + 0.41X, 



Q3- 



ERIC 



1 ^' / 



114 



Table 4.3 

Determination of Significant Student Elements Contributing 
to the Student Overall Ratings of Instructors 

Analysis of Variance 



Degrees of Freedom Sum of Squares Moan Squares 
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Regression 1 59.52 59.62 

Residual 4.97 208.96 0.42 

F - 141.80* 

Probability < 0.0001 level 

*F is significant 

Significance of Items From Student Elements Alone 

Item- Constant B t 

Q3, opinion of course—present -- 0.41 11.91 

k, constant 2.56 18.68 

p < 0.05 level with two-tailed test reached 

^Item wording has been shortened for table; the full text is included 
in Appendix F 

Data were sub-nitted to the multiple regression analysis to 
determine the significance of differences between the proportion of 
variance in student overall ratings of instructors explained by student 
ratings of any particular course item as compared to any other course 
item. The level of significance selected was the .05 level with a 
two-tailed test. The null hypothesis developed was as follows: 
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Null Hypothesis H(03): There is no significant difference 
between the proportion of variance in the student overall ratings of 
instructors explained by student ratings of one of the course elements 
as compared to each of the other course elements. 

H{03): ij{7) - u{8) = . . . - u{14). 

Results of the multiple regression analysis are included in 
Table 4.4. The calculated F was 106.34, which was significant at less 
than the .0001 level. Therefore, the null hypothesis H{03) was 
rejected and tha opposite established. There is a significant differ- 
ence between the proportion of variance in the student overall ratings 
of instructors explained by student ratings of one of the course 
elements as compared to each of the other course elements. 

Further analysis revealed that student ratings to three 
course items explained 95 percent of the variance in student overall 
ratings of instructors. These course items were Q8 . (the course was 
well planned and organized), Q9 (subject matter presented in the class- 
room agreed with the course objectives), and Q13 (instructional mate- 
rials/packets purchased in the bookstore or distributed in class were 
of what value in helping me understand the course). 

A regression formula was developed to determine the expected 
value (X) of item Q31 (the student overall ratings of the instructor) 
if the student ratings of significant course items w^-e known. The "B" 
coefficients are included in Table 4.4. The regression formula 
developed was the following: 

Xq3j - 1.52 + 0.45XgQ + 0.14Xgg + 0.05X^^3. 
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Table 4.4 

Determination of Significant Course Elements Contributing 
to the Student Overall Ratings of Instructors 

Analysis of Variance 



Degrees of Freedom Sum of Squares Mean Squares 

Regression 3 105.26 35.09 

Residual 495 163.33 0.33 

F = 106.34* 

Probability < 0.0001 level 

*F is significant 

Significance of Items From Course Elements Alone 

Item^ Constant B t 

Q9, course planning and organization -- 0.45 12.03 

Q8, subject matter agreer^-^t with 

objectives o.l4 3.16 

Q13, instructional materials' value 

in understanding course -- o.05 2.79 

k, constant 1.52 8.53 

p < 0.05 level with two-tailed test reached 



^Item wording has been shortened for table; the full text is included 
in Appendix F 

Data were submitted to the multiple regression analysis to 
determine the significance of the difference between the proportion of 
variance in student overall ratings of instrt:tors explained by student 
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ratings of any particular instructor element as compared to any other 
instructor element. The level of significance selected was the .05 
level with a two-tailed test. The null hypothesis developed was as 
f ol 1 ows : 

Null Hypothesis H{04): There is no significant difference 
between the proportion of variance in the student overall ratings of 
instructors explained by student ratings of one of the instructor 
elements as compared to each of the other instructor elements. 

H{04): u{15) ' u{16) - u{17) - . . . = u{30). 

Results of the multiple regression analysis are included in 
Table 4.5. The calculated F was 81.45, which was significant at the 
.0001 level. Therefore, the null hypothesis H{04) was rejected and the 
opposite established. There is a significant difference between the 
proportion of variance in the student overall ratings of instructors 
explained by student ratings of one of the instructor elements as com- 
pared to each of the other instructor elements. 

Further analysis revealed that student ratings to six instruc- 
tor items explained 95 percent of the variance in student overall 
rati"3s of instructors. These instructor items were Q23 (the 
instructor spoke in a way that helped me learn), Q22 (the instructor 
encouraged thinking, problem solving, and decision making), Q17 (the 
instructor explained material clearly), Q24 (the instructor 
communicated clearly in writing on the chalkboard, papers, and 
transparencies), Q28 (the instructor demonstrated interest in my 
learning), and Q16 (the instructor knew the subject well). 
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Table 4.5 

Determination of Significant Instructor Elements Contributing 
to the Student Overall Ratings of Instructors 

Analysis of Variance 



Degrees of Freedom Sum of Squares Mean Squares 



Regression 6 133.84 22.31 

Residual 4.92 134.75 0.27 

F - 81.45* 

Probability < 0.0001 level 

*F is significant 

Significance of Items From Instructor Elements Alone 

Item ^ Constant 6 t 
Q23, instructor spoke in a way to 

help me learn -- o.l9 4.15 

Q22, instructor encouraged thinking -- 0.15 3.74 

Q17, instructor explained material clearly -- 0.17 3.52 
Q24, instructor explained course 

information -- o.ll 3.11 

Q28, instructor interested in my learning -- 0.11 2.94 

Q16, instructor knew subject -- o.l5 2.33 

k, constant 0.29 -- 1.05 

p < 0.05 lei'el with two-tailed test reached 



Item wording has been shortened for table; the full text is included 
in Appendix F 
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A regression formula was developeu to determine the expected 
value (X) of item Q31 (the student overall ratings of the instructor) 
if student ratings of significant instructor items were known. The "B" 
coefficients are included in Table 4.5. The regression formula 
developed was the following: 

Xq31 - 0-29 + 0-19Xq23 + 0-15X^2. + O.^X^j^ + 0.11X^24 + 
0.11Xp28+0.15XQjg. 

Correlations Between Ratings of Items and 
Student Overall Ratings of Instructors 

The Pearson r correlations between student ratings of selected 
course, instructor, and student elements and student overall ratings of 
instructors were calculated. The ranking of correlations is indicated 
in Table 4.6. Correlations ranged from a high of +0.535 to a low of 
-0.089. 

There were two items that had correlations of over 0.50. They 
were Q23 (the instructor spoke in a way that helped me learn) and Q8 
(the course was well planned and rganized). Fourteen items had 
correlations between 0.40 and 0.60. Nine items had correlations 
between 0.20 and 0.40. Only one item, Q4 (course difficulty), had a 
negative correlation, which was a very negligible -0.089. 

Previous sections included th:^se items that explained 
significant proportions of the student overall ratings of instructors. 
Correlations of the remainder of the items are included in Table 4.6 so 
they can be examined. Examination of the correlations of several items 
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wi11 enable them to be used in developing and ranking goals in an 
effort to raise student rating levels. 

Table 4.6 

Ranking of Correlations of Student Ratings of Particular 
Course, Instructor, and Student Items With 
Student Overall Ratings of Instructors 



1 2 
Rank Items Pearson r 



1 


Q23, instructor spoke in a way to help 
me learn 


.636 


2 


Q8, course planning and organization 


.606 


3 


Q17, instructor explained material clearly 


.593 


4 


Q24, course difficulty 


.538 


5 


028. instructor intprPstpH in mv 1oav>ninn 

lllwl«IUV«l«UI IIILCICd LC\J 1 II HIV 1 cell li 1 Flu 


COR 


6 


Q22, instructor encouraged learning 


.510 


7 


Q19, instructor showed enthusiasm 


.500 


8 


Q21, instructor encouraged student participa- 
tion 


.483 


9 


Q20, instructor organized classes/labs 


.480 


10 


Q3, prejjent opinion of course 


.471 


11 


Q29, instructor application of course to life 
or work 


.459 


12 


Q25, instructor use of AVs 


.454 


13 


Q18, instructor demonstrated skills 


.450 


14 


Q9, subject matter agreement with objectives 


.429 


15 


Q30, instructor showed respect 


.403 


16 


Q16, instructor knew subject 


.400 
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Table 4.6 (Continued) 



Rank^ 


Items^ 


Pearson r 


17 


Q17, information regarding procedures was 
usable 




18 


Q26, timely return of assignments 


.376 


19 


QIO, assignment agreement with objectives 


.362 


20 


Q27, instructor comment,s regarding my 
assignments 


.345 


21 


Qll, evaluations agreement with objectives 


.340 


22 


QIS, instructor explanation of written 
course information 


337 


23 


Q13, instructional materials' value in 
understanding course 


.272 


24 


& y wCALUuuN^ Va 1 Uc HI unUcrd banu 1 nQ 

course 


.267 


25 


Q6, expected grade 


.209 


26 


Q14, course free from bias 


.144 


27 


Q2, initial opinion of course 


.041 


28 


Q5, weekly out-of-class preparation 


.031 


29 


Q4, course difficulty 


-.089 



Rank of one had the highest positive correlation 

"Item wording shortened for the table; the ^ull text is included in 
Appendix F 
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Comparison of Student Overall Ratings 
of Instructors by Groups 

Data were submitted to SPSS^ Scheffe' One-Way Analysis of 
Variance (ANOVA) to determine if there was a significant difference 
among/between student overall ratings of instructors by students in 
various groupings. The .05 level of significance was selected with a 
two-tailed test. The groups established for this part of the study 
were (1) function of course, (2) level of course, (3) instructional 
division, (4) age of student, (5) expected grade, (6) expression of 
course difficulty, (7) student opinion of course--initial , (8) student 
opinion of course- -present, (9) student out-of-class preparation, and 
(10) sex of student. 

ANOVA results are presented in the following sections. The 
means were based on a five-point scale. A "five" on the scale was con- 
sidered more positive than a "one" on the scale. 

Course Function (General Education vs. Occupational -Specific) 

Two of the four course functions were studied. The reason for 

this selection was that both functions of courses existed in all 

associate degree and vocational diploma programs offered at LTC. The 

two functions of courses selected were (1) general education and 

(2) occupational -specific. 

The null hypothesis was developed. The null hypothesis was 

tested at the .05 level using a two-tailed test. The hypothesis 

developed was as follows: 

er|c 1-G 
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Null Hypothesis H(05): There is nc significant difference 
between student overall ratings of instructors from students enrolled 
in a general education course and from those enrolled in an occupa- 
tional-specific course. 

H{05): u (general education) » u (occupational-specific). 

Table 4.7 

Co!^.parison of Means of Student 
riatings by Course Function 



Function 
of Course 



Standard Standard 95% Confidence 
Count Mean Deviation Error Interval for Mean 



Occupational - 
Speci fic 


267 


4.10 0.77 


0.05 


4.00 


to 4.19 


General 
Education 


190 


4.23 0.75 


0.06 


4.12 


to 4.33 


Total 


457 


' 15 0.77 




4.08 


to 4.22 






Analysis of Variance 






Source 




Degrees of 
Freedom 


Sum of 
Squares 


Meat. 
Squares 


F Ratio 


Between Groups 




1 


1.85 


1.85 


3.15 


Within Groups 




455 


256.74 


0.S9 




Total 




456 


268.53 







Critical F - 3.31* 

*F ratio is not signifi -ant 



ERIC 



124 

There v^ere 457 usable instrumerits collected from students in 
this grouping. Analysis indicated that 190 (42 percent) ratings were 
from stuuents enrolled in a general education course and 257 (58 
percent) ratings were from students enrolled in an occupational - 
specific course. Means and standard deviations were calculated and 
found to be 4.23 + C.75 for the ratings from students enrolled in a 
general education course and 4.10 + 0.77 for ratings from students 
enrolled in an occupational -specific course (see Table 4.7). Ratings 
in both groups were similar and relatively consistent. 

Analysis of variance calculations yielded a F ratio of 3.15 
(see Table 4.7). The critical F ratio ."or a two-tailed test with 
456 degrees of freedom is 3.81. The calculated F ratio was less than 
the critical F ratio. Therefore, the null hypothesis H(05) was not 
rejected. There was no significant difference between student overall 
Instructor ratings from students enrolled in a general education course 
and from those enrolled in an occupational -specific course. 

Course and Program Level 

Two levels of courses and programs are offered in full-time 
occupational programs at LTC. The two levels established were 
(1) associate degree and (2) vocational diploma. Courses in one level 
of program were not included as courses in another level of program. 

The null hypothesis was developed. The null hypothesis was 
tested at the .05 level of significance using a two-tailed test. The 
hypothesis developed was as follows: 
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Null Hypothesis H{06): There is no significant difference 
between student overall ratings of instructors from students enrolled 
in an associate degree-level course and from those enrolled in a voca- 
tional diploma-level course. 

H{06): u (associate degree) - u (vocational diploma). 

Table 4.8 

Comparison of Means of Student Ratings 
by Course and Program Level 



Level of 
Course 



Standard Standard 95% Confidence 
Count Mean Deviation Error Interval for Mean 



Associate 
Degree 

Vocational 
Diploma 

Total 



302 4.15 0.70 0.04 4.07 to 4.23 

155 4.15 0.88 0.07 4.01 to 4.30 

457 4.15 0.77 0.04 4.08 to 4.22 



Analysis of Variance 



Source 


Degrees of 
Freedom 


Sum of 
Squares 


Mean 
Squares 


Between Groups 


1 


0.00 


0.00 


Within Groups 


455 


268.58 


0.59 


Total 


456 


268.58 




Critical F » 3.C3* 

*F ratio is not significant 







F Pstio 
0.01 
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There were 457 usable instruments corected from students in 
this grouping. There were 302 (66 percent) ratings from students 
enrolled in associate degree-level courses and 155 (34 percent) 
responses from students enrolled in vocational diploma- ievel courses. 
Means and standard deviations were calculated and found to be 4.15 + 
0.70 for overall ratings of instructors from students enrolleu in 
associate degree-level courses and 4.15 + 0.88 for ratings from stu- 
dents enrolled in vocational diploma-level courses (see Table 4.8). 
Responses were almost identical with both being equal to the omposite 
mean with consistency slightly better for the associate degree group. 

Analysis of variance calculatioiis yielded a F ratio of 0.01 
(see Table 4.8). The critical F ratio for a two-tailed test with 
456 degrees of freedom was 3.81. The calculated F ratio was less than 
the critical F ratio. The null hypothesis H{06) was not rejected. 
There is no significant difference between student overall instructor 
ratings from students enrolled in an associate degree-level course and 
from those enrolled in a vocational diploma-level course. 

Instructional Division 

Students involved in xhe study were gro'jped according to 
instructional division. The study included students in four 
instructional divisions at LTC. These divisions were M) Business and 
Marketing, (2) Health Occupations, (3) Home Economics, and (4) Trade 
and Industry. 
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The null hypothesis was developed. The null hypothesis was 
tested at the .05 level of significance with a two-tailed test. The 
hypothesis developed was as follows: 

Null Hypothesis H{07): There is no significant difference 
between student overall ratings of instructors from students in the 
various divisions. 

H(07): u (i) . u (j); where i j; i, j = Business and 
Marketing, Health Occupations, Home Economics, or Trade and Industry. 

There were 441 usable instruments collected from students in 
this grouping. There were 203 (46 percent) from the Business and 
Marketing Division, 73 (17 percent) from the Health Occupations Divi- 
sion, 42 (10 percent) from the Home Economics Division, and 123 
(28 percent) from the Trade and Industiy Division. The data were tabu- 
lated, and calculations made and are included in Table 4.9. 

Means and standard deviations were calculated for each divi- 
s on. Division means of ratings ranged from a high of 4.50 to a low of 
4.01, with a composite mean of 4.15. Division means from highest to 
lowest were Home Economics--^ 50, Business and Marketing--4. 16, Health 
Occupations--*. 07, and Trade and Industry--4.0l . While all means wer^ 
in the 4.00 to 4.50 rangt, there was a considerable difference between 
the lowest three means and the highest mean. 

Analysis of variance calculations yielaad a F ratio of 4.64 
(see Tabli, 4.9). The critical F ratio for a two-tailed test with 
440 degrees of freedom was 3.97. The calculated F ratio was greater 
than the critical F ratio. Therefore, the null hypothesis H(07) was 
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Table 4.9 

Comparison of Means of Student Ratings 
by Instructional Division 



Division 



Standard Standard 95% Confidence 
Count Mean Deviation Error Interval for Mean 



Business and 
Marketing 203 

Health 

Occupations 73 

Home Economics 42 
Trade and 

Industry 123 

Total 441 



4.16 

4.07 
4.50 

4.01 
4.13 



1 



0.73 0.05 4.06 to 4.26 

0.89 0.10 3.86 to 4.28 

0.59 0.09 4.31 to 4.69 

0.76 0.07 3.87 to 4.14 

0.77 0.04 4.06 to 4.21 



Analysis of Variance 



Source 
Between Groups 
Within Groups 
Total 



Degrees of 
Freedom 

3 

437 
440 



Sum of 
Squares 

8.00 

251.11 

259.11 



Mean 
Squares 

2.67 

0.57 



F Ratio 
4.64 



Critical F - 3.97* 
*F ratio is significant 



1 



Significantly higher than the Health Occupations and Trade and 
Industry divisions 
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rejected and the opposite accepted. There is a significant difference 
between student overall ratings of instructors from students in two or 
more divisions. Student overall ratings of instructors from students 
in the Home Economics Division were significantly higher than ratings 
from students in the Trade and Industry and Health Occupations Divi- 
sions. 

• 

Student Age 

Students involved in the study were grouped according to age. 
Three age groups were established. These included (1) sixteen to 
twenty-two years, (2) twenty-three to twenty-nine years, and (3) over 
twenty-nine years of age. 

The null • hypothesis was developei. The null hypothesis was 
tested at the .05 level of significance with a two-tailed test. The 
hypothesis developed was as follows: 

Null Hypothesis H(08): There is no significant difference 
between student overall ratings of instructors from students in the 
various age groups. 

H(08): u (1) - u (j); where i j; i, j = eighteen to twenty- 
two years, twenty-three to twenty-nine years, or over twenty-nine 
years . 

There were 457 usable instruments collected from students in 
this grouping. There were 251 (55 percent) from the sixteen to twenty- 
two age group, 102 (22 percent) from the twenty- three to twenty-nine 
age group, and 104 (23 percent) from the o'er twenty-nine age group. 
Data were tabulated, and calculations made and included in Table 4.10. 
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Table 4.10 

Comparison of Means of Student 
Ratings by Student Age 



Age of 




Standard Standard 


95% Confidence 


Student 


r niiMT 

l«U UN it 


ncaii ucv 1 a L 1 Uii t rrur 


inLervai tor nean 


(Years of AopI 








16 to 22 


251 


4.12 0.79 0.05 


4.02 to 4.22 


23 to 29 


102 


4.06 0.73 0.07 


3.91 to 4.20 


Over 29 


104 


4.32 0.74 0.07 


4.17 to 4.46 


Total 


457 


4.15 0.77 0.04 


4.08 to 4.22 






Analysis of Variance 


• 


Source 




Degrees of Sum of 


Mea'ii 




Freedom Squares 


Squares F Ratio 


Between Groups 




2 3.99 


2.00 3.42 


Within Groups 




454 264.59 


0.58 


Total 




456 268.58 




Critical F 


05 ' ^• 


47* 




*F Ratio is not significant 





Means and standard deviations were calculated for each age 
group. Group means ranged from a high of 4.32 to a low of 4.06, with a 
composite mean of 4.15. Means from highest to lowest were over twenty- 
nine years age group--4.32, sixteen to twenty-two years age oroup-- 
4.12, and twenty-three to twenty-nine years age group--4.06. Students 
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in the over twenty-nine years age group rated instructors somewhat 
higher than students in the other two age groups. 

Analysis of variance calculations yielded a F ratio of 3.42 
(see Table 4.10). The critical F ratio for a two-tailed test with ^56 
degrees of freedom was 3.47. The calculated F ratio was 3.42. The 
null hypothesis H{08) was not rejected. There is no significant dif- 
ference between student overall instructor ratings from students in the 
various age groups. 

Student Expected Grade 

Students involved in the study were grouped according to the 
grade they expected to receive. Five groups of expected grade were 
established. These included A, B, C, I, and Don't Know. 

The null hypothesis was developed. The null hypothesis was 
tested at the .05 level of significance with a two-tailed test. The 
hypothesis developed was as follows: 

Null Hypothesis H{09): There is no significant difference 
between student overall ratings of instructors from students included 
in the various expected grade grcups. 

H{09): u (i) = u (j); where 1 ;^ j; i , j = A, B, C, I, or Don't 

Know. 

There were 454 usable instruments collected from students in 
this grouping. There were 139 (31 percent) collected ^'rom the "A" 
group, 173 (38 percent) collected from the "B" group, 98 (22 percent) 
collected from the "C" group, 7 (2 percent) collected from the- "I" 
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group, and 32 (7 percent) collected from the "Don't Know" group. Data 
were tabulated, and calculations made and included in Table 4.11. 

Table 4.11 

Comparison of Means of Student Ratings 
by Student Expected Graie 



Expected Standard Standard 95% Confidence 

Gi'ade Count Mean Deviation Error Interval for Mean 

Don't Know 32 3.89 0.81 0.13 3.62 to 4.16 

I 7 3.71 0.95 0.35 2.83 to 4.59 

C 98 3.93 0.77 0.08 3.77 to 4.08 

B 173 4.19 0.76 0.06 4.08 to 4.30 

A 139 4.36^ 0.68 0.06 4.25 to 4.47 

Total 454 4.15 0.77 0.04 4.08 to 4.22 

Analysis of Variance 

Degrees of Sum of Mean 

Source Freedom Squares Squares F Ratio 

Between Groups 4 14.99 3.75 6.73 

Within Groups 449 2EJ.22 0.56 

Total 453 265.21 



Critical F = 4.37* 
*F ratio is significant 



Significantly higher than the "C" and the "Don't Know" groups 
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Means and standard deviations were calculated for each group. 
Group means ranged from a high of 4.36 to a low of 3.71 with a compos- 
ite mean of 4.15. Means from highest to lowest were "A" group--4.36, 
"B" group--4.19, "C" group--3.93, "Don't Know" group--3.89, and "I" 
group--3.71. The means were progressively lower from the "A" group 
through the "I" group with the mean of the "Don't Know" group approxi- 
mately the same as that of the "C" group. 

Analysis of variance calculations yielded a F ratio of 6.73 
(see Table 4.11). The critical F ratio for a two-tailed test with 
453 degrees of freedom was 4.37. The null hypothesis H{09) was 
rejected and the opposite accepted. There is a significant difference 
between student overall ratings of instructors from students in two or 
more of the expected grade groups. Studenl overall ratings of instruc- 
tors from students indicating an "A" grade were significantly higher 
than ratings from those indicating a "C" grade or "Don't Know." 

Student Expression of Course Difficulty 

The students involved in the study were grouped according to 
their expressed difficulty level of the course. Five difficulty levels 
were established. These included (1) very difficult, (2) difficult, 
(3) average, (4) easy, and (5) very easy. 

The null hypothesis was developed. The null hypothesis was 
tested at the .05 level of significance with a two-tailed test. The 
hypothesis developed was as follows: 
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Null Hypothesis H(10): There is no significant difference 
between student overall ratings of instructors from students in the 
various expression of course difficulty groups. 

H{10): u (i) « u (j); where i / j; i, j = very difficult, 
difficult, average, easy, or very easy. 

There were 457 usable instruments collected from students in 
this grouping. There were 18 (4 percent) collected in the "very easy" 
group, 31 {7 percent) collected in the "easy" group, 222 (49 percent) 
collected in the "average" group, 157 (34 percent) collected in the 
"difficult" group, and 29 (6 percent) collected in the "very difficult" 
gruup. The data were tabulated, and calculations made and included in 
Table 4.12. 

Means and standard deviations were calculated for each group. 
Group means ranged from a low of 3.79 to a high of 4.23 with a compos- 
ite mean being 4.15. Means from highest to lowest were average- -4. 23, 
easy--4.16, very easy--4 11, difficult--4.10, and very difficult--3.79. 
Ratings from students in the average difficulty group were the highest. 

Analysis of variance calculations yielded a F ratio of 2.43 
(see Table 4.1^). Tha critical F ratio for a two-tailed test with 
456 degrees of freedom was 4.37. The null hypothesis H{10) was not 
rejacted. There is no significant difference between student overall 
ratings of instructors from students in the various expression of 
course difficulty groups. 
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Table 4.12 

Comparison of Means of Student Ratings 
by Expressed Difficulty 



Standard Standard 95% Confidence 
Difficulty Count Mean Deviation Error Interval for Mean 



Very Easy 


18 


4.11 


0.83 


0.20 


3.70 


to 


4.53 


Easy 


31 


4.16 


0.58 


0.10 


3.95 


to 


4.38 


Average 


222 


4.23 


0.70 


0.05 


4.14 


to 


4.33 


Difficult 


157 


4.10 


0.78 


0.06 


3.98 


to 


4.23 


Very Difficult 


29 


3.79 


1.11 


0.21 


3.37 


to 


4.22 


Total 


457 


4.15 


0.77 


0.04 


4.08 


to 


4.22 






Analys 


is of Vari 


ance 









Source 


Degrees of 
Freedom 


Sum of 
Squares 


Mean 
Squares 


Between Groups 


4 


5.66 


1.42 


Within Groups 


452 


262.92 


0.58 


Total 


456 


268.58 





F Ratio 
2.43 



Critical F = 4.37* 

*F ratio is not significant 



Student Opinion of Cours£--Initial 

Students involved in the study were grouped according to 
student's initial opinion of the course. Five opinion levels were 
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established. They were (1) highly negative, (2) negative, (3) no 
opinion, (4) positive, and (5) highly positive. 

The null hypothesis was developed. The null hypothesis was 
tested at the .05 level of significance using a two-tailed test. The 
hypothesis developed was as follows: 

Null Hypothesis H(ll): There is no significant difference 
bet vien student overall ratings of instructors from students in the 
various student initial opinion of the course groups. 

H(ll): u *i) = u (j); where i / j; i, j = highly negative, 
egative, no opinion, positive, or highly positive. 

There were 456 usable instruments collected from students in 
this grouping. There were 10 (2 percent) ratings from the "highly 
negative" group, 43 (9 percent) from the "negative" group, 121 
(27 percent) from the "no opinion" group, 205 (45 percent) from the 
"positive" group, and 77 (17 percent) from the "highly positive" group. 
The tabulations and calculations are included in Table 4.13. 

Means and standard deviations were calculated for each group. 
Group means ranged from a high of 4.25 to a low of 3.99 with a com- 
posite mean of 4.15. The means from highest to lowest were negative-- 
4.25, highly positive--4.22, highly negative- -4. 20, positiva--4.20, and 
neutral --3.99. 

Analysis of variance calculations yielded a F ratio of 1.85 
(see Table 4.13). The critical F ratio for a two-tailed test with 
455 degrees of freedom was 4.37. The null hypothesis H{11) was not 
rejected. There is no significant difference between the student 
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overall ratings of instructor«; from students in the various student 
initial opinion of the course groups. 

Table 4.1? 

Comparison of Means of Student 
Ratings by Initial Opinion 



Standard Standard 95% Confidence 
Opinion Count Mean Deviation Error Interval for Mean 



Highly Negative 


10 


4.20 


0.63 


0.20 


3.75 


to 


4.65 


llCV^Ci Live 




4.26 


0.69 


0.11 


4.04 


to 


4.47 


Neutral 


121 


3.99 


0.80 


0.07 


3.85 


to 


4.14 


Positive 


:o5 


4.20 


0.73 


0.05 


4.10 


to 


4.30 


Highly j'usitive 


77 


4.22 


0.85 


0.10 


4.03 


to 


4.41 


Total 


456 


4.15 


0.77 


0.04 


4.08 


to 


4.22 






Analysis of Variance 








Source 




Degrees of 
Freedom 


Sum of 
Squares 


Mean 
Squares 




F Rati 


Between Groups 




4 




4.34 


1.09 




1.85 


Wi c.iin Groups 




451 




264.22 


0.59 






Total 




455 




268.56 








Critical F 


05-*- 


73* 













*F ratio is not significant 
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Student Opinion of Course- -Present 

Students involved in the study were grouped according to stu- 
dent's present opinion of the course. Five opinion levels were estab- 
lished. They were: (1) highly negative, (2) negative, (3) no opinion, 
(4) positive, and (5) highly positive. 

The null hypothesis was developed. The null hypothesis was 
tested at the .05 level of significance using a two-tailed test. The 
hypothesis developed was as follows: 

Null Hypothesis H(12): There is no significant diff»>rence 
between the student overall ratings of instructors from students in the 
various student present opinion of course groups. 

H(12): u (i) - u (j); where i / j; i, j « highly negative, 
negative, no opinion, positive, or highly positive. 

There were 455 usable instruments collected from students in 
this grouping. There were 4 (1 percent) collected from the "highly 
negative" group, 33 (7 percent) from the "negative" group, 50 
(11 percent) from the "no opinion" group, 258 (59 percent) from the 
"positive" group, and 101 (22 percent) from the "highly positive" 
group. The data were tabulated, and calculations made and included in 
Table 4.14. 

Means and standard deviations were calculated for each group. 
Means ranged from a high of 4.5i to a low of 2.75. Means wsre pro- 
gressively lower as the opinion was lowe. ed from highly positive (4.51) 
to highly negative (2.75). This difference in means was verv large as 
compared to differences in the other groupings included In i.ie study. 



ERIC 



152 



139 



Table 4.14 

Comparison of Means of Student 
Ratings by Present Opinion 



Standard Standard 95% Confidence 
Opinion Count Mean Deviation Error Inter/al for Mean 



Highly Negative 


4 


2.75 


1.71 


0.85 


0.03 


to 


5.47 


Negative 


33 


3.24 


0.83 


0.14 


2.95 


to 


3.54 


Neutral 


50 


3.68 


0.74 


0.10 


J • *f / 


LO 




Positive 


268 


4.19^ 


0.65 


0.04 


4.12 


to 


4.27 


Highly Positive 


101 


4.61^ 


0.55 


0.54 


4.51 


to 


4.72 


Total 


456 


4.15 


0.77 


0.04 


4.08 


to 


4.22 






Analysis of Variance 








Source 




Degrees of 
Freedom 


Sum of 
Squares 


Mean 
Squares 




F Ratio 


Between Groups 




4 




68.32 


17.08 




38.60 


Within Groups 




451 




199.54 


0.44 






Total 




455 




267.86 









Critical F » 4.37* 
*F ratio is significant 



Significantly higher than means of the "Highly Negative", "Negative," 
and "Neutral" groups 

Significantly higher than means of all other groups 
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Analysis of variance calculations yielded a F ratio of 38.60 
(see Table 4.14). Tlie critical F for a two-tailed test with 455 
degrees of freedom was 4.37. The null hypothesis H{12) was rejected 
and the opposite accepted. There is a significant difference between 
the student overall ratings of instructors from students in the various 
student present opinion of course groups. The mean of ratings from 
students in the "highly positive" opinion group was significantly 
higher than the means of ratings from students in all other opinion 
groups. The mean of ratings from students in the "positive" opinion 
group was significantly higher than the means of ratings from students 
in the "neutral," "negative," or "highly negative" opinion of the 
course groups. 

Student Out-of-Class Preparation 

Students involved in the study were grouped according to the 
student's out-of-class preparation in hours per week. Five levels of 
preparation were established: (1) zero hours, (2) one to five hours, 
(3) five to ten hours, (4) ten to fifteen hours, and (5) over fifteen 
hours. 

The null hypothesis was developed. The null hypothesis was 
tested at the .05 level of significance using a two-tailed test. The 
hypothesis level oped was as follows: 

Null Hypothesis H{13): There is no significant difference 
between the student overa''l instructor ratings from students in the 
various out-of-class preparation groups. 
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K(13): u (i) = •( (j); where i ^ j; i, j = 0, 1-5, 5-10, lC-15, 
or over IS hours per week. 

Table 4.15 

Comparison of Maans of Stulent 
Ratings by Preparation T:?ne 



Preparation 
Time 


Count 


Mean 


Standard Standard 
Deviation Error 


95% Confidence 
Interval for Mean 


(Hours per Week) 










0 


15 


3.92 


0.80 0.21 


3.49 to 4.38 


1 to 5 


229 


4.14 


0.76 0.05 


4.05 to 4.24 


5 to 10 


146 


4.20 


0.72 0.06 


4.09 io 4.32 


iO to 15 


48 


4.02 


0.96 0.14 


3.74 to 4.30 


Over 15 


19 


4.37 


0.68 0.16 


4.04 to 4.70 


Total 


457 


4.15 


0.77 0.04 


4.08 to 4.22 






Analysis of Variance 




Source 




Degrees of Sum of 
Freedom Squares 


Mean 

Squares F Ratio 



Between Grcups 
Within Groups 
Total 



4 

452 
456 



2.76 
265.82 
268.58 



0.69 
0.59 



1.16 



Criti-al F = 4.37* 

*F ratio is not significant 
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There were 457 usable instruments collected from students in 
this grouping. There were 15 (3 percent) collected from the "z ro 
hours per week" group, 229 (50 percent) from the 'one tj five hours per 
week" group, 146 (32 percent) from the "five to ten hours per week" 
group, 48 (11 percent) from the "ten to fifteen hours per week" group, 
and 19 (4 percent) from the "over fifteen hours per week" group. The 
data were tabulated, and calculations made and included in Table 4.15 

Means and standard deviations ware calculated for tach group. 
Means ranged from a high of 4.37 to a low of 3.93 with a composite mean 
of 4.15. Means were lowered as out-of-class prpparation was reduced, 
except for the "ten to fifteen hours per week" group. The mean for 
this group was betv^een the means of the "five to ten hours per week" 
group and the "zero hours per week" group. 

Analysis of variance calculations yielded a F ratio of 1.15 
(see Table 4.15). The critical F for a two-tailed test with 456 
degrees of freedom was 4.37. The null hypothesis H(13) was not 
rejected. There is no significant difference between the student 
overall ratings of instructors from students in the various out-of- 
class preparation groups. 

Student Gender 

Students involved in the study were grouped according to gender 
of the student, female or male. 

The null hypothesis was developed. The null hypothesis was 
tested at the .05 level of significance with a two-tailed test. The 
hypothesis developed was as follows: 
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Null Hypothesis H(14): There is no significant difference 
between student overall ratings of instructors from female students and 
ratings from male students. 

H(14): u (females) « u (males). 

Tab^e 4.16 

Comoarison of Means of Student 
Ratings by Student Gender 



Standard Standard 95% Confidence 



Sex 


Count 


Mean Deviation Error 


Interval 


for Mean 


Male 


153 


4.04 0.76 0.64 


3.92 to 


4.16 


Femal e 


303 


4.21 0.76 J. 44 


4.12 to 


4.30 


Total 


456 


4.15 0.76 0.04 
Analysis of Variance 


4.08 to 


4.22 


Source 




Degrees of Sum of 
Freedom Squares 


Mean 
Squares 


F Ratio 


Between Groups 




1 3.01 


3.01 


5.17 


Within Groups 




454 264.25 


0.58 




Total 




455 267.26 






Critical F 


05 ==3. 


81* 







^F ratio is significant 



There were 456 usable instruments collected from students in 
this grouping. There were 303 f66 percent) ratings from the "female 



ERIC 



15? 



144 



stuoant" group and 153 (34 percent) ratings from the "male student" 
group. The data were tabulated, and calculations made and included in 
Table 4.16. 

The mean of the ratings from female students was 4.21. The 
mear» of the ratings from male students was 4.04. The standard 
deviation for both groups was an id*^ntical 0.76, indicating similar 
consistency in the ratings from both groups. 

Analysis of variance calculations yielded a F ratio of 5.17 
(see Table 4 16). The critical F ratio for a two-tailed test with 
455 degrees of freadom was 3.81. The null hypothesis (H14) was 
rejected and the opposite accepted. There is a significant difference 
between i.tudent overall ratings of instructors from female students and 
ratings from mal'j students. The overall ratings of instructors from 
female students are significantly higher than those from male students. 



CHAPTER 5 

SUMMARY, INTERPRETATION, CONCLUSIONS, AND RECOMMENDATIONS 



The purpose of this study was to refine a part of the instruc- 
tional assessment procedures at LTC. This was accu.vl ished by obtain- 
ing and analyzing data for course, instructor, and student elements to 
determine their relative relationships to student overall ratings of 
instructors. Equally important, this study also determined the biasing 
pffects of particular course, instructor, and student elements; student 
demographics; and college course and program classifications on student 
overall ratings of instructors. This is important in providing 
information necessary to enable LTC staff to make adjustments and 
interpret the data in a comparable manner regardless of course netting. 
Adjustments made in interpreting ratings are prerequisite to making 
decisions about development and ranking of goals for an instructor's 
individual professional growth plan. Chapter Five includes the follow- 
ing five sections: (1) Overview, (2) Interpretdtion of Results, 

(3) Applications of Findings to Future Instructor Assessments, 

(4) Conclusions, and (5) Recommendations. 

Overview 

There has been in recent years a considerable increase in the 
number of postsecondary two-year educational institutions implementing 
instructor professional growth programs that emphasize growth of the 
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ind- idual. This has resulted from an increased emphasis on 
accountability and a reduction in nonstructured courses of professional 
growth. These programs are becoming more dependent on assessments to 
determine strengths and weaknesses of the individual. Development and 
ranking of goals to be included in the instructor's professional growth 
program are based on interpretations of data adjusted as a result of 
the determination of significant course, instructor, and student 
elements; student demographics; or institutional course and program 
classifications. 

Although three types of instructional assessment are used, peer 
and supervisor assessments appear to have had rather limited usage. 
The student type of instructional assessment has become the dominant 
assessment used as the basis for identifying, developing, and ranking 
goals in instructor growth programs. Its acceptance and use by both 
administrative and instructional staff are increasing. 

There are both philosophical and practical reasons for the 
continued increase in use of student ratings of elements. Philosophi- 
cally, it can be concluded that the students are most affected by the 
instructor and that the instructor has the most contact with them for a 
substantial period of tine. One practical reason is that a significant 
positive correlation between student achievement and ratings given to 
the instructor seems to exist. Other practical reasons include avail- 
ability of computerized data entry terminals and ease in development of 
college-specific assessment instruments to assist in assessment and 
analysis of data. Literature supports the importance of using 
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locally-oriented, diagnostic-type assessment instruments to gain 
insights into the instructor's teachi.ng performance. Literature also 
indicates a need to localize results and to use those results in 
instructor growth program decisions if an acceptable and effective 
program is to be achieved. 

The LTC Student Assessment of Course and Instructor instrument 
was designed in accordance with the suggesv.ions made in the literature. 
It is diagnostic in nature. It also contains items within course, 
instructor, and si ident elements suggested to have an effect on s ;udent 
overall ratings of instructors and, thereby, student achievement. This 
study was an attempt to determine biasing influences of student ratings 
of particular elements, student demographics, and course and program 
classificatio'is on student overall ratings of instructors at the local 
level and appropriate adjustments to be made to localize the ratings 
for the particular course and students. This was considered necessary 
if interpretations of data were 1:0 serve as the basis for any develop- 
ment and ranking of goals for inclusion in the instructor's profes- 
si'^ial growth program. 

Results of this study should provide the collene with data and 
analyses to facilitate adjustments to data before making interpreations 
and decisions regarding instructor goals. Use of ratings appropriately 
adjusted in accordance with significant elements, student demographics, 
and institutional course and program classifications will enable deci- 
sions to be based on specific .instructional assignments. This will 
make data appropriate for decisions about instructor growth regardless 
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of differences in these elements, demographics, or classifications. 
The growth program wiM then have a greater chance to be successful. 

To illustrate this adjustment, the following example is used* 
Assume that overall ratings of instructors from the femaie students 
were found to be significantly higher than those from male students. 
The ratings for an instructor having a large proportion of females 
would then have to be adjusted downwaru by a factor to make these 
ratings comparable; or for those instructors having a large proportion 
of males, the rating would b^ adjusted upward by a factor to make these 
ratings comparable to data obtained in other courses. The instruc- 
tional administrative staff would then have to make the decision to 
adjust, the rating value upward for instructors teaching mostly male 
students or downward for those teaching female students. 

The determination of amount of adjustment necessary would be 
made for each group found to be significant in the study. The amount 
of adjustment to be made would be communicated to ;''l instructional 
staff. This would enable each person reviewing student ratings data 
analysis to make similar adjustments and thereby have a common base for 
interpretations. 

A number of studies included in Chapter Two indicated that 
student ratings of instructors have high correlations with student 
achievement. A high ranking would be given to goals based on elements 
having a significant relationship with the overall ratings of instruc- 
tors. Therefore, indirectly, a high priority is placed on goals that 
will probably contribute most toward increasing student achievement. 
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Another example is also presented here. Assume that the item 
(course was planned and organized) was found to have a high positive 
correlation with the student overall ratings of instructors, and that 
the item (assignments agreed with course objectives) was found to have 
a low correlation with the student overall ratings of instructors. The 
instructor receiving low ratings from students on both items would 
address both when developing growth goals. However, because of the 
higher correlation and level of significance with the student overall 
ratings of instructors, tne goal developed to improve the planning and 
organization of the course would be given a higher priority in the 
instructor's professional growth plan. 

In setting priorities for goals and activities for an instruc- 
tor's individual professional growth program, both the administrator 
and instructor would need to know the course, instructor, and student 
items that had significant relationships with the student overall 
ratings of instructors by ranking and selecting goals based on correla- 
tions. Instructors will be including those that have the greatest 
potential for improving student overall ratings of instructors and 
student achievement. 

Interpretation of Results 

Relationships of Ratings of Items With 
Student Overall Ratings of Instructors 

Research Question 1 ; Is there a relationship between student 
overall ratings of instructors and student ratings of i*ems in the 
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three major element groups (course, instructor, and student) in the 
assessment of instruction at LTC? If so, what is the nature of the 
relationship? 

The composite mean (4.15) of the overall ratines of instructors 
item was above average (see Table 4.1). This speaks well of LTC 
instructors included in the study. Courses were selected according to 
occupational program and not by instructor criteria. Therefore, the 
mean would probably have been similar if all the instructors had been 
included. The high, mean value indicates that relatively high caliber 
instructors currently teach at the college. 

Results indicated that the items within instructor elements 
were given high ratings by students. The item (instructor :;new 
subject) received the highest rating (see Table 4.1). It was. indicated 
previously that subjoct area competence wr given the highest weight 
during the hiring process. Student ratings supported -jse of that 
criterion in the hiring of instructors and also its empiiasis during the 
previous instructor professional growth plans. 

Student ratings of instructor knowledge of subject seemed to be 
independent of other items. Ratings of instructor's level of knowledge 
had correlations with other items that ranged from none (-.023) to low 
(+.455) (see Appendix J). The standard deviation of only 0.43 indi- 
cated that students seemed vo judge instructor knowledge level quite 
uniformly. 

Analysis of the data (see Table A.?.) indicated that student 
ratings to only five items (instructor spoke in a way that helped me 
learn; course was well planned and organized; instructor encouraged 
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thinking, problem solving, and decision making; instructor explained 
materials clearly; and instructor demonstrated interest in my learning) 
were found to have a significant relationship with the student overall 
ratings of instructors. Therefore research question one is answered 
as follows: There is a significant difference between the relation- 
ships of the student ratings of various elements with the student over- 
all ratings of instructors. This is based on analysis indicating that 
ratings to only five of the twenty-nine items included in the ftudy had 
a significant relationship. 

Ratings for no student-related item (for example, out-of-class 
preparation) were found to have a significant ralationship with the 
student overall ratings of instructors when all items were analyzed. 
When items within the student-related item grouping were a«ialyzed 
separately, only one (opinion of the course--present) was found to have 
a significant relationship with student overall ratings of instructors. 
Therefore, indications are that student-related elements have very 
little relationship with student overall ratings of instructors. 

Relationship of Items, Characrarist ics. 
an d Classificationi~on St - ent 
Overall Ratings of Instructors 

Research Qiestion 2 ; Is them a relationship between student 
overall ratings of instructors and student ratings of particular items 
within each of the three major element groups, student de-nographics, or 
place of the course in college academic programming? 

The biasing effect of particular elements (course, instructor, 
and student), student demographics, and course and program classifica- 
tions on student ratings of instructors has "-een the subject of 
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numerous reseat-ch studies reported in the literature. Results of these 
studies were varied and nconclusive and, therefore, not transportable 
to other institutions. The results of this study provided an answer to 
the question that was applicable to the college. Significant differ- 
ences were found in overall ratings of instructors from students in the 
following groups: instructional division of course, grade expected by 
student, student's present opinion of ourse, and sex of student. The 
. extent of biasing irfluence of each of these groups and groups of other 
elements, student demographics, and course and program classifications 
if. further discussed in this section. 

C cirs" Function . There was no significant difference between 
the overall ratings of instructors from those students in the general 
education courses and from those students if, occupational -specific 
courses (see Table 4.7). Instructional managers speculated that 
courses in these two functions would probably yield the most diverse 
student ratings. Results did not support this. Although the mean for 
overall ratings of instructors from students in general education 
courses was slightly higher than the mean from ratings of students in 
occupational courses, the difference w?s not significant. 

One caution needs to be made at this point in the interpreta- 
tion of results. Only 71 percent of the number of students who 
completed the instrument for an occupational course completed the 
instrument for a general education course. The resc of the students 
received advanced standing for the general education course, completed 
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the course early, dropped the course, or for some other reason did not 
complete the instrument. 

Results indicated that overall ratings of instructors from 
students in general education courses tended to be slightly higher than 
ra ngs in the occupational -specific courses. This difference was not 
atistically significant; therefore, this fact is not to be considered 
when adjusting student ratings of instructors. This classification 
will not be included in selecting and ranking goals for the 
instructor's individual professional growth program. 

Course and Program Level . There was no significant difference 
between the overall ratings of inst,. jctors from students in the. 
associate degree-level courses and the ratings from students in the 
vocational diploma-level courses (see Table 4.8). The means were 
identical. This element is not to be considered when adjusting student 
ratings of instructors and will not be a factor to be used in selecting 
and ranking goals for the instructor's individual professional growth 
program. 

Instructional Division , nverall ratings of instructors from 
students in the Home Economics Division were substantially higher than 
those ratings from students in the three other divisions (see Table 
4.9) Ratings from students in the Home Economics Division wc^-*; sig- 
nificantly higher than ratings from students in the Health occupations 
Division and the Trade and Industry Division. The ratings from stu- 
dents n this group will require an adjustment to make them comparable 
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with ratings from students in other instructional divisions. Ratings 
from students in the Home Economics Division programs will need to be 
adjusted downward to enable decisions regarding ranking of goals to be 
based on comparable information. 

Student Age . Over one-half of the students in the study were 
In the sixteen to twenty-two years age group (see Table 4.10). One- 
fourth were in each of the other age groups, twenty-three to twenty- 
nine years and over twenty-nine years age groups. Some slight 
differences were observed among the groups. Ratings from students in 
the over twenty-nine years age group had a considerably but not 
significantly higher mean than ratings from students in the other two 
groups. Therefore, adjustments for age group need not be made in 
student overall ratings of instructors. 

Student Expected Grade . Approximately 30 percent of the stu- 
dents responding expected an "A" grade, 40 percent a "B" ^rade, and 
20 percent a "C" grade (see Table 4.11). Only 9 percent expected an 
"I" grade or were unsure of their grade. There was a significant 
difference in these overall ratings of instructors from students in 
particular expected grade groups. Students expecting an- "A" grade 
rated the instructor significantly higher than those expecting a "C" 
gi^dde or those who did not know. The small percentage expecting an "I" 
grade made it difficult to interpret that reK .ionshIp with the overall 
ratings of instructors. Therefore, adjustments in overall ratings of 
instructors would need to be made in an upward direction if a 
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prepondei afice of students in the course were expecting a "C" grade tr 
were in a "Don't Know" status when the assessment form was 
administered. The adjustment would need to be made before 
interpretations of ratings were used in developing and ranking goals 
for the instructor's professional growth program. 

St udent Expression of Course Difficulty . Approximately 50 per- 
cent of the students indicated that courses were of "average diffi- 
culty," and 35 percent indicated that courses were "difficult" (see 
Table 4.12). Remaining ratings were about evenly distributed among the 
other three categories. Responses from the "average difficulty" group 
of students tended to have higher overall instructor ratings. The 
overall rating^ of instructors were progressively lower on each side of 
the "average difficulty" level. 

Differences in student overall ratings of instructors among the 
difficulty levels were not significant. Therefore, no adjustments need 
to be made in the -atings before interpretation of student overall rat- 
ings of instructors for this grouping. 

Student Opinion of Course--Initial . Approximately 25 percent 
of the students indicated "no opinion," and 45 percent indicated a 
"positive opinion" of the course before students enrolled (see Table 
4.13). Slightly less than 20 percent indicated a "highly positive" 
rating.. This indicated a substantial skew in the number of ratings 
toward the "positive opinion." Differences in student overall ratings 
of instructors among the particular groups indicated that initial 
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opinion of course was not a significant biasing influence. Therefore, 
no adjustments need to be made in the ratings before interpretation of 
student overall ratings of instructors for this grouping. 

Student Opinion of Course—Present . Slightly more than 80 per- 
cent of the ratings in this group were "positive" or "highly positive" 
{see Table 4.14). This indicates that a large proportion of the 
students believed courses were meeting their needs. Slightly more than 
10 percent had "no opinion" or were neutral about the course. Only 
8 percent indicated a "negative" feeling about the courses they were 
taking. 

The large range of means from 4.61 to 2.75 (see Table 4.14) 
indicated tha direct correlation between present opinion of the course 
and the overall ratings of instructors with the mean of ratings from 
the "highly negative" opinion student group being the lowest. The high 
F ratio (3^.60) indicated a significant difference in overall ratings 
of instructors from students in the various present opinion level 
groups. Ratings from two groups, "highly positive'' and "positive," 
were significantly higher than those from groups having lower present 
opinions of the course. Ratings of present opinion of the course are 
definitely to be considered in adjusting student ratings of instructors 
before their interpretation. They are also to be used when -anking 
goals for the instructor's professional growth program if the mean of 
the opinion ratings is not at or above the expected level for the 
instructor. 
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Student Out-of-Class Preparation . One-half of the students 
indicated an out-of-cUss preparation in the range of "one to five" 
hours per week, and one-third indicated theirs to be in the range of 
"five to ten" hours per week (see Table 4.15). Other ratings were 
distributed among the three other groups. One factor that would 
potentially be a concern in interpreting this item is that courses 
having a variety of levels and credits were involved in the study. 

Overall ratings of instructors tended to be more positive as 
the student put in more out-of-class preparation time. However, 
differences between ratings from students in the various groups were 
not significant. Therefore, no adjustments need to be made for this 
element when interpreting the overall ratings of instructors. Out-of- 
class preparation need not be taken into consideration when ranking 
goals to be developed in the instructor's professional growth program. 

Student Gender . There were twice as many female students as 
malp students in the study (see Table 4.16). The overall ratings of 
instructors from female student? were somewhat higher than ratings of 
male students. Consistency of ratings from both groups was the same as 
a standard deviation o^ " 76 was obtainad for both. The inean (4.21) of 
ratings from female st jants was found to be significantly higher than 
the mean (4.04) of ratings from male students. An adjustment to the 
werall ratings of instructors needs to be made for the gender of 
students whenever there is a preponderance of one gender in the course. 
The adjustment would be downward if there were substantially more 
females than males in the course and upward if there were substantially 
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nnre males than females in the course. Consideration of gender of 
student would not be included in developing and ranking goals for the 
instructor's professional growth program. 

Applications of Findings to Future 
Instructor Assessments 

The findings have useful applications to future interpretations 
of student assessments of Instructors. Results indicated that biasing 
influences of particular elements, student demographics, and course and 
program classifications did exist. There is no reason to believe that 
they will not continue to exist. Therefore, the following direct 
applications of the findings are suggested. 

Female students rated instructors significantly higher than did 
male students. Therefore, adjustments to the ratings need to be made 
whenever substantial gender imbalance exists. The adjustment would be 
downward if there were substantially more females than males in the 
course and upward if there were substantially more males than females 
in the course. 

Students indicating "positive" end "highly positive" present 
opinions of the course rated instructors significantly higher than 
those indicating lower opinion levels. Therefore, an adjustment needs 
to be made to the ratings whenever the opinion levels are substantially 
different from those norip-.lly expressed for courses offered by the 
colleye. The adjustment would be downward if there was a substantially 
higher opinion and upward if thera was a substantially lower opinion. 
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Students expecting an "A" grade rated instructors significantly 
higher than those expecting a "C" grade and those anticipating no par- 
ticular grade. Therefore, an adjustment needs to be made to the rat- 
ings whenever th** distribution of expected grade is substantially 
diffore.it from the distribution normal for the college. The adjustment 
would be downward whenever there is a substantially larger proportion 
of students expecting an "A" grade and upward whenever there is a 
substantially larger portion of students expecting a "C" grade or 
anticipating no parti ;ular grade. 

Students in the Home Economic: Division rated instructors 
significantly higher than students in other instructional divisions. 
There re, adjustment needs to be made to the ratings whenever the 
composition of students in the course from each instructional division 
is substantially different from the actual proportion of the college 
student body from each division. The adjustment would be downward if 
there was a higher than actual proportion of students from the Home 
Econohi' :s Division in the course and upward if there was a lower than 
actual proportion from the Home Economics Division. 

Use of studei.c ratings of items within course and instructor 
elements as a basis for developing goals fo> the instructor's profes- 
sional growth plan is an important emphasis in LIFE. The process for 
using the ratings is as follows. First, items for which student 
ratings are substantially lower than desired (4.0) are identified. 
Second, identified items are separated into two groups--those items 
found to have a significant relationship with overall ratings of 
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instructors in this study and those that did not. Third, all items are 
ranked with the item having the highest correlation ("B") with the 
overall ratings of instructors given a rank of number one. Fourth, the 
Instructor and administrator select items for use in developing goals. 
Those selected include all the significant relationship items and 
additional not significant ones that are reasonable for the effort to 
be expended. The additional items are selected according to their 
rank. And last, goals are developed based on the selected items for 
inclusion in the instructor's professional growth program. 

Conclusions 

The major conclusions reached as a result of this study were as 

follows: 

1. There was strong support at LTC fcr the use of formal 
student ratings as a reasonable and reliable way of measuring the level 
of instructor performance and for collecting assessment data for use in 
developing and ranking goals for the individual instructor profes- 
sional growth program. 

2. Students tended to rate course and instructor elements and 
instructor overall performance in a consistent and discriminating way. 

3. Student gender influenced ratings; overall ratings of 
instructors by female students were significantly higher than those by 
male students. 

4. Overall ratings of instructors by students in the various 
instructional divisions were significantly different. Ratings by 
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students in the Home Economics Division were significantly higher than 
ratings by students in other divisions, 

5, Differences in overall student ratings of instructors were 
not explained evenly by responses to the items in the course, 
instructor, and student elements included in the study; relatively few 
show a significant capability in this respect, 

6, Student ratings of the course element item (course planning 
and organization) tended to explain differences in the student overall 
ratings of instructors- 

7, Student ratings of instructor element items (instructor 
spoke in a way to help me learn, instructor encouraged thinking, 
instructor explained material clearly, and instructor interested in my 
learning) also had significant relationships with student overall 
ratings of instructors, 

8, In the student element group, student ratings of two items 
(expected grade and present opinion of course) were important; both 
were significantly related to student overall ratings of instructors, 

9, In contrast, student ratings of the student element items 
(initial opinion of course, out-of-class effort, and difficulty) showed 
them to be not important; they were not significantly related to 
student overall ratings of instructors, 

10, None of the other twenty-four items in the course, 
instructor, and student elements was important; none provided signifi- 
cant explanations of differences in student overall ratings of 
instructors. 
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11- Age of student was not an important factor; it was not 
significantly related to student overall ratings of instructors, 

12, Nor were institutional classifications of course function 
and course level found to be important; neither was significantly 
related to student overall ratings of instructors* 

Recommendations 

Recommendations for Use of Results 

The following recommendations were developed from the results 
of the study: 

L The college should continue to use student ratings of 
course, instructor, and student elements and overall ratings of 
instructors to provide data for developing and ranking goals for the 
instructor's professional growth program, 

2- The college should be encouraged to continue use of the 
Student Assessment of Instruction instrument to ob^-ain student ratings 
of course, instructor, and student elements and student overall ratings 
of instructors. 

3- The college staff should make an appropriate adjustment to 
student ratings whenever there is an atypical distribution of students 
in a course in terms of students' expectation of at^ades, expressions of 
difficulty, and/or present opir?ions of course, or the presence of dis- 
balanced ratios of female to male students, as well as the instruc- 
tional divisions in the course. 
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4. A task force should be organized to determine the extent of 
the adjustment to be made when each of the foregoing conditions is 
found. It should communicate the adjustment levels throughout the 
college so similar interpretations will result when the same set of 
student ratings prevail. 

5. Highest weight should be given to the student ratings to 
the five items (course planning and organization, instructor spoke in a 
way to help me learn, instructor encouraged thinking, instructor 
explained material clearly, and instructor interested in my learning) 
in the instructor element group found in the study to be important 
Influencing factors during the development and ranking of primary goals 
for the instructor's professional growth program proceeds. 

6. Development and ranking of secondary goals related to items 
found in the study not to be important influencing factors for the 
instructor's professional growth program should be based on comparisons 
of correlations ("8") between student ratings of items and overall 
ratings of the instructor. Only those elements i 'presented by items 
having adjusted student rating means substantially lower than 4.0 need 
be considered. 

7. Additional studies -.hould be conducted using additional 
occupational programs at the college to further validate the results of 
this study. 

8. Additional studies should be conducted using subsequent 
courses in the sequence of courses in each occupational program to 
further validate the results of this study. 
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9. Periodic studies should be conducted to monitor the 
validity of the relationships and biasing effects of the various 
elements, st- ient demc .phics, and institutional course and program 
classifications on student ratings as additional occupational programs 
emerge and as the student body makeup changes. 

Plans for Diffusion and Implementation 

The writer has plans to present this study's results and recom- 
mendations to the following agencies and groups: 

1. The Lakeshore Technical College Board of Education. 
Members will be informed of result: of the study to obtain necessary 
s«pport to continue use of the Student Assessment of Instruction 
instrument to collect data for use in decisions regarding development 
of goals for instructor professional growth proorams. 

2. The Lakeshore Technical College faculty and management 
staff. They are to be informed of results so that they might support 
the I'se of the Student Assessment of Instruction instrument for 
collecting data, making adjustments before interpreting the data, and 
using interpretat .ons to identify goals for the instructor's 
professional growth program. 

3. The Wisconsin Vocational, Technical and Adult Education 
Administrators-Research, Planning, and Development Committee. This 
committee is composed of research representatives from each of the 
sixteen districts and state board starf members. Committee members are 
Interested in research methodology and also have the responsibility for 
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disseminating results of research studies conducted in one district or 
by state staff members to staff membei - ^n the respective districts. 

4. The Wisconsin Vocational, Technical and Adult Education 
Administrators—Instructional Services Committee. This coi.miittee is 
composed of instructional representatives from each of the sixteen dis- 
tricts and state board staff members. Committee members are interested 
in information related to the betterment of instruction. Results of 
this study will be helpful to committee members in identifying instruc- 
tor gro.;th goals at their respective colleges. 

5. The instructional administrators of various secondary and 
tiostsecondary schools throughout the state of Wisconsin. Results will 
be incorporated into the LIFE program and disseminat to various 
admi. strators at instructional and faculty -^valuation meetings. 

6. The Wisconsin Board of Vocational, T.chnical and Adult 
Education sponsored Evaluation Conference participants. Participants 
attend to learn new techniques or o-jt^omes obtai ad through us<i of 
evaluation in the district for evaluating district prog'-ams, services, 
and staff. 

7. The readers of professional journals, periodicals, and 
local newspapers. Results and recommendations of this study will be 
sup»marized and submitted for publication in the following journals, 
ptriodicals, and newspapers. 

a- Wisconsin Educational Kasearch Association Newsletter . 
This is a state association of researchers in educational insti tut : -'>s 
at all levels. Articles provide an opportunity to share research 
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reihodology and results. Members will be able to share results with 
Instructional managers at their respective educational institutions and 
with students in teacher training and educational administration pro- 
grams . 

b. The Wisconsin Vocational Educator . This periodical is 
published by tiid Vocational Studies Center- -Univprsity of Wisconsin- 
Madison for the purpose of disseminating information and results of 
studies that have potential benefit to vocational educators in its 
service area. 

c- Community and Junior College Journal . This is the pro- 
fessional journal of the American Association of Co-nmunity and Junior 
Colleges. Articles in it provide an opportunity to share the results 
of the study with research and instructional managers in two-year 
colleges. 

d. The driftwood . Tnis is a publication distributed by 
the college to LTC staff and secondary school administrators in the 
area. Articles in this publication describe to all staff members tyoes 
of activities being conducted at the college. 

e. Manitow^c-Herald-Times and The Sheboygan Press . These 
are official LTC newspapers. Publication in these newspapers will 
provide ar opportunity to publicize LTC efforts to provide a relevant 
basis for instructor professional growch programs designed to increase 
Instructional performance levels. 

This study was designed to obtain and analyze data to determine 
(1) the biasing infliences of student ratings on particular items 
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included in course, instructor, and student elements; student 
demographics; and college course and program classifications on student 
overall ratings of instructors and (2) the relationships among student 
ratings of pa^ Jcular course, instructor, and student elements on the 
student overall ratings of instructors. Results can be useful in 
enabling instructional staff members to make appropriate adjustments 
before interpreting student rating data for instructors teaching 
coursas in a variety of settings. It is anticipated that results of 
this study will have a positiv*? effect on the acceptance of instructor 
assessment data collected through the use of the Student Assessment of 
Instruction instrument. It should also have ' positive effect on 
acceptance and use of assessment data in developing, ranking, and 
selecting goals for an instructor's professional growth program. 
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The Self 
Actualization Process 
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FOR EXCELLENCE 



ERIC 



191 



20 



APPENDIX G 

PROGRAMS, COURSES, AND NUMBER OF STUDENTS 
INCLUDED IN THE STUDY BY DIVISION 
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ASSOCIATE DEGREE PROGRAMS 



Business and Marketing Divis i on 



Program 

Administrative 
Assistant-- 
Information 
Processing 

Administrative 

Assistant-- 

Secretarial 

Court and Con- 
ference Report- 
ing 

Data Processing 



Marketing 



Materials 
Management 



Number of 
Students 

18 



1* 



25 



14 



Occupational 
Course 

Information 
Processing Con- 
cepts 



Information 
Processing Con- 
cepts 

Machine Short- 
hand I 



Data Processing 
Concepts 

Marketing Prin- 
cipiles 



Overview of 
Management 



General Educa- 
tion Course 

Office Communi- 
cations I 



Office Communi- 
cations I 



Office Communi- 
cations I 



Communication 
Skills I 

Business and 
Professional 
Speech 

Communication 
Skills I 



Health Occupations Division 

10 



Associate Degree 
Nursing 

Dispensing 
Optician 
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Nursing I 



Ophthalmic 
Optics 



Human Growth and 
Development 

Communication 
Skills I 



Trade and Industry Division 

Electrical Power 10 DC Fundamentals Economics 

Engineering 

Technician 
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Trade and Industry Division (continued) 



Program 

Electromechani- 
cal Technology 

Electronics 
Technician 

Mechanical 
Design Techni- 
cian 

Plastics Tech- 
nician 



Number of 
Students 

13 
10 
10 

11 



Occupational 
Course 

Basic Elec- 
tricity 

DC Fundamentals 



Dimensions and 
Working Drawings 



Introduction to 
Plastics 



General Educa- 
tion Course 

Communication 
Skills I 

Communication 
Skills I 

Communication 
Skills I 



Communication 
Skills I 



VOCATIONAL DIPLOMA PROGRAMS 



business and Marketing Division 
Office Assistant 18 



Word Processing 
Special ist 



11 



Clerical 
Accounting 

Electronic Word 
Processing 



Communications 
Communications 



Health Occupations Division 

Pharmacy Tech- 15 
nical Aide 



Drug Classifica- Basic Communica- 
tion tions 



Home Economics Division 

Child Care 
Services 
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Art Activities 



Basic Communica- 
tions 



Trade and Industry Division 
Graphic Arts 18 



Lithography 
Theory 



Graphic Commu- 
nications I 
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APPENjIX h 

INSTRUCTIONS FOR ADMINISTERING THE STUDENT 
ASSESSMENT OF INSTRUCTION INSTRUMENT 
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1. Tell students that the purpose of the responses is to provide 
information to the instructor for use in improving the course 
organization and content delivery. It will take approximately 15 
minutes to complete the form, 

Z. Tell students to use a pencil to complete the form. 

3, Tell students to complete the following information at the top of 
the form: 

a. Instructor Name: Write in the instructor's name, 

b. Course Number: Write the number across the top, and darken 
the digits below each part of the written number. 

c. Year: Darken "87" (as indicated), 

d- Term: Darken "1" for fall, "2" for spring, "3" for summer, or 
as designated. 

e. Age: Write the student's age across the top, and darken the 
digits below each of the written numbers, 

f. Sex: Darken either "F" or "M," 

g. Program: Write the number across the top (get the number from 
Program Identification for Use), and darken the digits below 
each part of the written number. 

4, Tell students to completely darken the rectangle with the letter 
of the choice that agrees with their opinion for numbers 1 through 



5, Collect the forms when students have completed them, 

6, Check the form to make sure rectangle spaces have been completely 
filled in. 

7, Send the completed forms to Marv Schrader, 
Thank you for your assistance in this assessment. 
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APPENDIX I 

MEANS AND STANDARD DEVIATIONS OF STUDENT RATINGS 
OF ITEMS 2 THROUGH 31 ON STUDENT ASSESSMENT 
OF INSTRUCTION INSTRUMENT 
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standard 

Item Mean Deviation 

1 Not used in study, 

2 My opinion of the course before it 

started was: 3,64 ,94 

3 My opinion of the course at present is: 3,93 .85 

4 1 found this course to be (difficulty): 3,32 ,84 

5 My weekly out-of-class preparation for 

this course has averaged: 2.64 .90 

6 The grade I expect to receive in this 

course is: 3,82 L13 

7 Information regarding course procedures, 
expectations, and outcomes distributed in 

writing at the start of the course was: 4.20 .74 

8 The course was well planned and organ- 
ized: 3.99 .83 

9 Subject matter presented in the classroom 

agreed with the course objectives: 4.51 .68 

10 Assignments agreed with course objec- 
tives: 4,63 .64 

11 Exams/evaluations agreed with course 

objectives: 4,60 .66 

12 The required text(s) was/were of what 
value in helping me understand the 

course? 3.98 1.02 

13 Instructional materials/packets purchased 
in the bookstore or distributed in class 
were of what value in helping me under- 
stand the course content? 3.56 1.38 

14 Course content and materials were free 

from bias: 4.75 .61 
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Standard 

Item Mean Deviation 

15 The instructor explained writtan course 
information, goals of course, attendance 

policy, and grading policy: 4.69 .78 

16 The instructor knew the subject well. 4.79 .43 

17 The instructor explained material 

clearly. 4.73 .75 

18 The instructor demonstrated course- 
related skills well. 4.52 .70 

19 The instructor showed enthusiasm for the 

subject. 4.66 .61 

20 The instructor organized classes/labs for 

effective use of time. 4.40 .78 

21 The instructor encouraged student par- 
ticipation in class or lab activities. 4.48 .81 

22 The instructor encouraged thinking, prob- 
lem solving, and decision making. 4.23 .73 

23 The instructor spoke in a way that helped 

me learn. 4.27 .90 

24 The instructor communicated clearly in 
writing on the chalkboard, papers, and 

transparencies. 4.33 .90 

25 The instructor made appropriate use of 

teaching aids. 4.17 1.00 

26 The instructor graded and returned 
assignments, exams, and quizzes within a 

reasonable time. 4.57 .73 

27 The instructor provided comments on my 

completed assignments. 4.11 1.20 

28 The instructor demonstrated interest in 

my learning. 4.28 .83 

29 The instructor helped me apply course 

information to life or work. 4.06 .99 



o 21 

ERLC 



200 



Standard 

Item Mean Deviation 

30 The instructor treated me with respect. 4.55 .57 

31 My overall rating of the instructor is: 4.15 .73 
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APPENDIX J 

CORRELATIONS OF STUDENT RATINGS OF ITEMS WITH 
STUDENT OVERALL INSTRUCTOR RATINGS 
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Item Correlation 

1 Not used in study. 

2 My opinion of the course before it started was: .041 

3 My opinion of the course at present is: .471 

4 I found this course to be (difficulty): -.089 

5 My weekly out-of-class preparation for this 

course has averaged: .031 

6 The grade I expect to receive in this course is: .209 

7 Information regarding course procedures, expec- 
tations, and outcomes distributed in writing at 

the start of the course was: ,390 

8 The course was well planned and organized: .606 

9 Subject matter presented in the classrocn agreed 

with the course objectives: .429 

10 Assignments agreed with course objectives: .362 

11 Exams/evaluations agreed with course objectives: .340 

12 The required text{s) was/were of what value in 

helping me understand the course? .267 

13 Instructional materials/packets purchased in the 
bookstore or distributed in class were of what 
value in helping me understand the course con- 
tent? .272 

14 Course content and materials were free from 

bias: .144 

15 The instructor explained written course informa- 
tion, goals of course, attendance policy, and 

grading policy: .337 

IP The instructor knew the subject well. .397 
17 The instructor explained material clearly. .593 
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Item 



Correlation 



18 The instructor demonstrated course-related 
skills well • 

19 The instructor showed enthusiasm for the 
subject. 

20 The instructor organized classes/labs for effec- 
tive use of time. 

21 The instructor encouraged student participation 
in class or lab activities. 

22 The instructor encouraged thinking, problem 
solving, and decision making. 

23 The instructor spoke in a way that helped me 
learn. 

24 The instructor communicated clearly in writing 
on the chalkboard, papers, and transparencies. 

25 The instructor made appropriate use of teaching 
aids. 

26 e instructor graded and returned assignments, 
exams, and quizzes within a reasonable time. 

27 The instructor provided comments on my completed 
assignments. 

28 The instructor demonstrated interest in my 
learning. 

29 The instructor helped me apply course informa- 
tion to life or work. 

30 The instructor treated me with respect. 

31 My overall rating of the instructor is: 



.450 
.500 
.480 
.483 
.510 
.636 

• 538 

• 454 
.376 

• 345 
.525 

• 459 
.403 
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BIOGRAPHICAL SKETCH 



Marvin A. Schrader was born in Medford, Wisconsin, and soon 
after moved to Dorchester, Wisconsin. He graduated from the Dorchester 
High School in May of 1954. The farm background led him to enroll in 
the Agriculture Education program at the University of Wisconsin-River 
Falls graduating with a baccalaureate degree having majors in Agricul- 
ture Education and Chemistry Educafon in 1958. 

After graduation, he accepted a contract to teach secondary 
education at Fairchild High School, Fairchild, Wisconsin. While there, 
he taught the subjects of agriculture, biology, chemistry, driver 
education, and physics. He also served as assistant basketball coach. 

In 1961, he received a National Science Foundation (NSF) grant 
to attend a science education program at the University of North 
Dakota. During the year following the grant, he worked as a laboratory 
assistant in the National Science Foundation Education program while 
continuing studies for a degree. He received the Master of Science 
Teaching degree (a degree unique to the University of North Dakota) 
with majors in Chemistry and Physics and a minor in Biology. 

Upon receiving the master's degree, he accepted a" chemistry 
teaching position at Lincoln High School in Manitowoc, Wisconsin. 
While teaching, he became interested in new approaches and techniques 
in the teaching field. Interested in the area of questioning tech- 
niques, he assisted Dr. Norris Sanders in the development work for the 
book. Classroom Questions, What Kinds (Harper and Row, 1966). While 
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teaching at Lincoln High School, he was also awarded two National 
Science Foundation summer grants to (1) become acorainted with the 
philosophy and mechanics of teaching the CHEM Bond Approach high school 
chemistry course and (2) develop laboratory r.:iterials for the CHEM Bond 
Approach course. He also rec&iveu a National Defense Education Act 
(NDEA) summer grant to study research in education. 

He, with four other staff members from the Manitowoc Public 
School System,, wrote a curriculum development proposal and i^eceived a 
federal grant to fund a Cooperative tlucational Service Agency (CESA) 
#10 Curriculum Development Center for a four-year period. He then 
served in the capacity of mathematics and science curriculum consultant 
at the center with responsibilities for assisting kindergarten through 
grade twelve parochial and public school teachers in the incorporation 
of the new mathematics and science approaches into their curriculum and 
objectives and questioning techniques into their teaching. This was 
accomplished through workshop involvement and consultation with 
individual teachers. 

After the project ended, he began employment as the first 
curriculum specialist for the Lakeshore Vocational, Technical and Adult 
Education District (LTI). In this capacity, he introduced and assisted 
in implementing the curriculum development techniques utilized in 
previous years in the kindergarten through grade twelve level to the 
postsecondary level vocational, technical, and adult education instruc- 
tional staff. The position responsibilities involved both curriculum 
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development and staf development activities. He is currently employed 
as a research and curriculuir^ specialist at LTC* 

Marvin has shared his experience and knowledge with educators 
and noneducators in a number of ways, A number of workshops 
emphasizing measurable objectives and questioning techniques have been 
presented at colleges a-^d universities, A sampling includes Gustavus 
Adolphus College, Mankato State College, and Texas Christian 
University, He also serves as a University of Wisconsin-Stout adjunct 
instructor teaching the course "Course Construction," 

A number of presentations have been made at regional and state 
educational group conferences and meetings on the topics of curriculum 
development, staff development, and teaching techniques. He has also 
authored or coauthored a number of papers and reports on topics ranging 
from curriculum articulation to providing alternative delivery systems 
for providing education/training to employees in noneducational 
institutions. 

To increase his competence level, Marvin enrolled in the 
Educational Special ist--Vocational Education program at University of 
Wisconsin-Stout in Menomonie, Wisconsin, and was awarded the 
Educational Specialist degree in 1974, He enrolled in the Nova 
University Doctor of Education Program i/ Higher Education in 1981, 

Marvin presently holds state of Wisconsin lifetime licenses in 
secondary educati on as a teacher of agri cul ture , bi ol ogy , and 
chemistry. He also holds Wisconsin VTAE System five-year certification 
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as an instructional specialist. These have been awarded on the basis 
of course work, educational experience, and work experience. 

An interest in professional organization participation is 
illustrated by the life membership he holds in both the American 
Vocational Association and the National Science Teachers Association. 
He holds charter memberships in the Lakeshore Vocational Association, a 
local of the American Vocational Association, and in the Society for 
the Promotion of Individual Worth in Education and Training and member- 
ships in the American Educational Research Association, Association for 
Supervision and Curriculum Development, and Wisconsin Vocational 
Association having also served as a board member. He serves as the 
Lakeshore VTAE District representative to the Wisconsin . Vocational , 
Technical and Adult Education Administrators Association Research, 
Planning, and Development Committee and recently completed a four-year 
sequence as secretary-treasurer, vice chairperson, chairperson, and 
past chairperson. 

Marvin and his wife, Janet, reside just outside the city of 
Sheboygan, Wisconsin, a city of approximately 50,000 people located 
along the west shoreline of Lake Michigan. They have three children. 
Daryl, the oldest, is an actuary for an insurance company. Cynthia is 
an administrative clerk in the United States Marine Corps at Camp 
Pendleton. David, the youngest, just graduateH from North High School 
in the Sheboygan Area School District. 



SIGNATURE PAGE 



2?2 



I certify that I have read and am willing to sponsor this Major 
Applied Research Project submitted by Marvin A. Schrader. In my 
opinion, it conforms to acceptable standards and is fully adequate in 
scope and quality as a Major Applied Research Project for the degree of 
Doctor of Education at Nova University. 



i Datel Sebastian V. Martorana, Ph.D. 



MARP Advisor 



I certify that I have read and am willing to sponsor this Major 
Applied Research Project; and in my opinion, it conforms to acceptable 
standards for a Major Applied Research Project for the degree of Doctor 
of Education at Nova University. 



7 Date ^ CI ete Hinton , M . D . 

Local Committee Member 



This Major Applied Research Project was submitted to the 

Central Staff of the Center for Higher Education of Nova University and 

is acceptable as a partial fulfillment of the requirements for the 
degree of Doctor of Education. 




^ Philip n. DeTurk, Ed.D. 
Central Staff Committee Member 
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